On the global convergence of a parameter-adjusting Levenberg-Marquardt method

Previous Article
Delay-range dependent $H_\infty$ control for uncertain 2D-delayed systems
NACO Home
This Issue
Next Article
Analysis of complexity of primal-dual interior-point algorithms based on a new kernel function for linear optimization

2015, 5(1): 25-36. doi: 10.3934/naco.2015.5.25

On the global convergence of a parameter-adjusting Levenberg-Marquardt method

Liyan Qi ^1, , Xiantao Xiao ^1, and Liwei Zhang ^1,

1.	School of Mathematical Sciences, Dalian University of Technology, Dalian 116023, China, China

Received January 2015 Revised March 2015 Published March 2015

Abstract
Related Papers

The Levenberg-Marquardt (LM) method is a classical but popular method for solving nonlinear equations. Based on the trust region technique, we propose a parameter-adjusting LM (PALM) method, in which the LM parameter $\mu_k$ is self-adjusted at each iteration based on the ratio between actual reduction and predicted reduction. Under the level-bounded condition, we prove the global convergence of PALM. We also propose a modified parameter-adjusting LM (MPALM) method. Numerical results show that the two methods are very efficient.

Keywords: Nonlinear equations, global convergence., Levenberg-Marquardt method.

Mathematics Subject Classification: Primary: 90C30, 65K0.

Citation: Liyan Qi, Xiantao Xiao, Liwei Zhang. On the global convergence of a parameter-adjusting Levenberg-Marquardt method. Numerical Algebra, Control & Optimization, 2015, 5 (1) : 25-36. doi: 10.3934/naco.2015.5.25

References:

[1]	J. Fan, The modified Levenberg-Marquardt method for nonlinear equations with cubic convergence,, Math. Comp., 81 (2012), 447. doi: 10.1090/S0025-5718-2011-02496-8.
[2]	J. Fan, Accelerating the modified Levenberg-Marquardt method for nonlinear equations,, Math. Comp., 83 (2014), 1173. doi: 10.1090/S0025-5718-2013-02752-4.
[3]	J. Fan and J. Pan, Convergence properties of a self-adaptive Levenberg-Marquardt algorithm under local error bound condition,, Comput. Optim. Appl., 34 (2006), 47. doi: 10.1007/s10589-005-3074-z.
[4]	J. Fan and Y. Yuan, On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption,, Computing, 74 (2005), 23. doi: 10.1007/s00607-004-0083-1.
[5]	K. Levenberg, A method for the solution of certain nonlinear problems in least squares,, Quart. Appl. Math., 2 (1944), 164.
[6]	D. W. Marquardt, An algorithm for least-squares estimation of nonlinear inequalities,, SIAM J. Appl. Math., 11 (1963), 431.
[7]	J. J. Moré, Recent developments in algorithms and software for trust region methods,, in Mathematical Programming: the state of the art (Bonn, (1983), 258.
[8]	J. J. Moré, B. S. Garbow and K. E. Hillstrom, Testing unconstrained optimization software,, ACM Trans. Math. Software, 7 (1981), 17. doi: 10.1145/355934.355936.
[9]	J. Nocedal and S. J. Wright, Numerical optimization,, 2nd edition, (2006).
[10]	N. Yamashita and M. Fukushima, On the rate of convergence of the Levenberg-Marquardt method,, in Topics in numerical analysis, 15 (2001), 239. doi: 10.1007/978-3-7091-6217-0_18.

show all references

References:

[1]	J. Fan, The modified Levenberg-Marquardt method for nonlinear equations with cubic convergence,, Math. Comp., 81 (2012), 447. doi: 10.1090/S0025-5718-2011-02496-8.
[2]	J. Fan, Accelerating the modified Levenberg-Marquardt method for nonlinear equations,, Math. Comp., 83 (2014), 1173. doi: 10.1090/S0025-5718-2013-02752-4.
[3]	J. Fan and J. Pan, Convergence properties of a self-adaptive Levenberg-Marquardt algorithm under local error bound condition,, Comput. Optim. Appl., 34 (2006), 47. doi: 10.1007/s10589-005-3074-z.
[4]	J. Fan and Y. Yuan, On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption,, Computing, 74 (2005), 23. doi: 10.1007/s00607-004-0083-1.
[5]	K. Levenberg, A method for the solution of certain nonlinear problems in least squares,, Quart. Appl. Math., 2 (1944), 164.
[6]	D. W. Marquardt, An algorithm for least-squares estimation of nonlinear inequalities,, SIAM J. Appl. Math., 11 (1963), 431.
[7]	J. J. Moré, Recent developments in algorithms and software for trust region methods,, in Mathematical Programming: the state of the art (Bonn, (1983), 258.
[8]	J. J. Moré, B. S. Garbow and K. E. Hillstrom, Testing unconstrained optimization software,, ACM Trans. Math. Software, 7 (1981), 17. doi: 10.1145/355934.355936.
[9]	J. Nocedal and S. J. Wright, Numerical optimization,, 2nd edition, (2006).
[10]	N. Yamashita and M. Fukushima, On the rate of convergence of the Levenberg-Marquardt method,, in Topics in numerical analysis, 15 (2001), 239. doi: 10.1007/978-3-7091-6217-0_18.

[1]	Jinyan Fan, Jianyu Pan. Inexact Levenberg-Marquardt method for nonlinear equations. Discrete & Continuous Dynamical Systems - B, 2004, 4 (4) : 1223-1232. doi: 10.3934/dcdsb.2004.4.1223
[2]	Jinyan Fan, Jianyu Pan. On the convergence rate of the inexact Levenberg-Marquardt method. Journal of Industrial & Management Optimization, 2011, 7 (1) : 199-210. doi: 10.3934/jimo.2011.7.199
[3]	Jinyan Fan. On the Levenberg-Marquardt methods for convex constrained nonlinear equations. Journal of Industrial & Management Optimization, 2013, 9 (1) : 227-241. doi: 10.3934/jimo.2013.9.227
[4]	Johann Baumeister, Barbara Kaltenbacher, Antonio Leitão. On Levenberg-Marquardt-Kaczmarz iterative methods for solving systems of nonlinear ill-posed equations. Inverse Problems & Imaging, 2010, 4 (3) : 335-350. doi: 10.3934/ipi.2010.4.335
[5]	Zhili Ge, Gang Qian, Deren Han. Global convergence of an inexact operator splitting method for monotone variational inequalities. Journal of Industrial & Management Optimization, 2011, 7 (4) : 1013-1026. doi: 10.3934/jimo.2011.7.1013
[6]	Wei-Zhe Gu, Li-Yong Lu. The linear convergence of a derivative-free descent method for nonlinear complementarity problems. Journal of Industrial & Management Optimization, 2017, 13 (2) : 531-548. doi: 10.3934/jimo.2016030
[7]	Chunlin Hao, Xinwei Liu. Global convergence of an SQP algorithm for nonlinear optimization with overdetermined constraints. Numerical Algebra, Control & Optimization, 2012, 2 (1) : 19-29. doi: 10.3934/naco.2012.2.19
[8]	Zhong Tan, Qiuju Xu, Huaqiao Wang. Global existence and convergence rates for the compressible magnetohydrodynamic equations without heat conductivity. Discrete & Continuous Dynamical Systems - A, 2015, 35 (10) : 5083-5105. doi: 10.3934/dcds.2015.35.5083
[9]	Zhiyou Wu, Fusheng Bai, Guoquan Li, Yongjian Yang. A new auxiliary function method for systems of nonlinear equations. Journal of Industrial & Management Optimization, 2015, 11 (2) : 345-364. doi: 10.3934/jimo.2015.11.345
[10]	Martin Burger, José A. Carrillo, Marie-Therese Wolfram. A mixed finite element method for nonlinear diffusion equations. Kinetic & Related Models, 2010, 3 (1) : 59-83. doi: 10.3934/krm.2010.3.59
[11]	Thierry Colin, Boniface Nkonga. Multiscale numerical method for nonlinear Maxwell equations. Discrete & Continuous Dynamical Systems - B, 2005, 5 (3) : 631-658. doi: 10.3934/dcdsb.2005.5.631
[12]	Carlos E. Kenig. The method of energy channels for nonlinear wave equations. Discrete & Continuous Dynamical Systems - A, 2019, 0 (0) : 1-15. doi: 10.3934/dcds.2019240
[13]	Nora Merabet. Global convergence of a memory gradient method with closed-form step size formula. Conference Publications, 2007, 2007 (Special) : 721-730. doi: 10.3934/proc.2007.2007.721
[14]	Walter Allegretto, Yanping Lin, Zhiyong Zhang. Convergence to convection-diffusion waves for solutions to dissipative nonlinear evolution equations. Conference Publications, 2009, 2009 (Special) : 11-23. doi: 10.3934/proc.2009.2009.11
[15]	José A. Carrillo, Jean Dolbeault, Ivan Gentil, Ansgar Jüngel. Entropy-energy inequalities and improved convergence rates for nonlinear parabolic equations. Discrete & Continuous Dynamical Systems - B, 2006, 6 (5) : 1027-1050. doi: 10.3934/dcdsb.2006.6.1027
[16]	Yang Li, Yonghong Ren, Yun Wang, Jian Gu. Convergence analysis of a nonlinear Lagrangian method for nonconvex semidefinite programming with subproblem inexactly solved. Journal of Industrial & Management Optimization, 2015, 11 (1) : 65-81. doi: 10.3934/jimo.2015.11.65
[17]	W. Wei, H. M. Yin. Global solvability for a singular nonlinear Maxwell's equations. Communications on Pure & Applied Analysis, 2005, 4 (2) : 431-444. doi: 10.3934/cpaa.2005.4.431
[18]	Daniela Giachetti, Maria Michaela Porzio. Global existence for nonlinear parabolic equations with a damping term. Communications on Pure & Applied Analysis, 2009, 8 (3) : 923-953. doi: 10.3934/cpaa.2009.8.923
[19]	Monica Conti, Elsa M. Marchini, V. Pata. Global attractors for nonlinear viscoelastic equations with memory. Communications on Pure & Applied Analysis, 2016, 15 (5) : 1893-1913. doi: 10.3934/cpaa.2016021
[20]	Zuzana Došlá, Mauro Marini, Serena Matucci. Global Kneser solutions to nonlinear equations with indefinite weight. Discrete & Continuous Dynamical Systems - B, 2018, 23 (8) : 3297-3308. doi: 10.3934/dcdsb.2018252

Impact Factor:

American Institute of Mathematical Sciences