American Institute of Mathematical Sciences

Advanced
  • Previous Article
    Analysis of complexity of primal-dual interior-point algorithms based on a new kernel function for linear optimization
  • NACO Home
  • This Issue
  • Next Article
    Delay-range dependent $H_\infty$ control for uncertain 2D-delayed systems
2015, 5(1): 25-36. doi: 10.3934/naco.2015.5.25

On the global convergence of a parameter-adjusting Levenberg-Marquardt method

Liyan Qi 1, , Xiantao Xiao 1, and  Liwei Zhang 1,

1. 

School of Mathematical Sciences, Dalian University of Technology, Dalian 116023, China, China

Received  January 2015 Revised  March 2015 Published  March 2015

The Levenberg-Marquardt (LM) method is a classical but popular method for solving nonlinear equations. Based on the trust region technique, we propose a parameter-adjusting LM (PALM) method, in which the LM parameter $\mu_k$ is self-adjusted at each iteration based on the ratio between actual reduction and predicted reduction. Under the level-bounded condition, we prove the global convergence of PALM. We also propose a modified parameter-adjusting LM (MPALM) method. Numerical results show that the two methods are very efficient.
Keywords: Nonlinear equations, global convergence., Levenberg-Marquardt method.
Mathematics Subject Classification: Primary: 90C30, 65K0.
Citation: Liyan Qi, Xiantao Xiao, Liwei Zhang. On the global convergence of a parameter-adjusting Levenberg-Marquardt method. Numerical Algebra, Control & Optimization, 2015, 5 (1) : 25-36. doi: 10.3934/naco.2015.5.25
References:
[1]

J. Fan, The modified Levenberg-Marquardt method for nonlinear equations with cubic convergence,, Math. Comp., 81 (2012), 447.  doi: 10.1090/S0025-5718-2011-02496-8.  Google Scholar

[2]

J. Fan, Accelerating the modified Levenberg-Marquardt method for nonlinear equations,, Math. Comp., 83 (2014), 1173.  doi: 10.1090/S0025-5718-2013-02752-4.  Google Scholar

[3]

J. Fan and J. Pan, Convergence properties of a self-adaptive Levenberg-Marquardt algorithm under local error bound condition,, Comput. Optim. Appl., 34 (2006), 47.  doi: 10.1007/s10589-005-3074-z.  Google Scholar

[4]

J. Fan and Y. Yuan, On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption,, Computing, 74 (2005), 23.  doi: 10.1007/s00607-004-0083-1.  Google Scholar

[5]

K. Levenberg, A method for the solution of certain nonlinear problems in least squares,, Quart. Appl. Math., 2 (1944), 164.   Google Scholar

[6]

D. W. Marquardt, An algorithm for least-squares estimation of nonlinear inequalities,, SIAM J. Appl. Math., 11 (1963), 431.   Google Scholar

[7]

J. J. Moré, Recent developments in algorithms and software for trust region methods,, in Mathematical Programming: the state of the art (Bonn, (1983), 258.   Google Scholar

[8]

J. J. Moré, B. S. Garbow and K. E. Hillstrom, Testing unconstrained optimization software,, ACM Trans. Math. Software, 7 (1981), 17.  doi: 10.1145/355934.355936.  Google Scholar

[9]

J. Nocedal and S. J. Wright, Numerical optimization,, 2nd edition, (2006).   Google Scholar

[10]

N. Yamashita and M. Fukushima, On the rate of convergence of the Levenberg-Marquardt method,, in Topics in numerical analysis, 15 (2001), 239.  doi: 10.1007/978-3-7091-6217-0_18.  Google Scholar

show all references

References:
[1]

J. Fan, The modified Levenberg-Marquardt method for nonlinear equations with cubic convergence,, Math. Comp., 81 (2012), 447.  doi: 10.1090/S0025-5718-2011-02496-8.  Google Scholar

[2]

J. Fan, Accelerating the modified Levenberg-Marquardt method for nonlinear equations,, Math. Comp., 83 (2014), 1173.  doi: 10.1090/S0025-5718-2013-02752-4.  Google Scholar

[3]

J. Fan and J. Pan, Convergence properties of a self-adaptive Levenberg-Marquardt algorithm under local error bound condition,, Comput. Optim. Appl., 34 (2006), 47.  doi: 10.1007/s10589-005-3074-z.  Google Scholar

[4]

J. Fan and Y. Yuan, On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption,, Computing, 74 (2005), 23.  doi: 10.1007/s00607-004-0083-1.  Google Scholar

[5]

K. Levenberg, A method for the solution of certain nonlinear problems in least squares,, Quart. Appl. Math., 2 (1944), 164.   Google Scholar

[6]

D. W. Marquardt, An algorithm for least-squares estimation of nonlinear inequalities,, SIAM J. Appl. Math., 11 (1963), 431.   Google Scholar

[7]

J. J. Moré, Recent developments in algorithms and software for trust region methods,, in Mathematical Programming: the state of the art (Bonn, (1983), 258.   Google Scholar

[8]

J. J. Moré, B. S. Garbow and K. E. Hillstrom, Testing unconstrained optimization software,, ACM Trans. Math. Software, 7 (1981), 17.  doi: 10.1145/355934.355936.  Google Scholar

[9]

J. Nocedal and S. J. Wright, Numerical optimization,, 2nd edition, (2006).   Google Scholar

[10]

N. Yamashita and M. Fukushima, On the rate of convergence of the Levenberg-Marquardt method,, in Topics in numerical analysis, 15 (2001), 239.  doi: 10.1007/978-3-7091-6217-0_18.  Google Scholar

[1]

Jinyan Fan, Jianyu Pan. Inexact Levenberg-Marquardt method for nonlinear equations. Discrete & Continuous Dynamical Systems - B, 2004, 4 (4) : 1223-1232. doi: 10.3934/dcdsb.2004.4.1223
[2]

Jinyan Fan, Jianyu Pan. On the convergence rate of the inexact Levenberg-Marquardt method. Journal of Industrial & Management Optimization, 2011, 7 (1) : 199-210. doi: 10.3934/jimo.2011.7.199
[3]

Jinyan Fan. On the Levenberg-Marquardt methods for convex constrained nonlinear equations. Journal of Industrial & Management Optimization, 2013, 9 (1) : 227-241. doi: 10.3934/jimo.2013.9.227
[4]

Johann Baumeister, Barbara Kaltenbacher, Antonio Leitão. On Levenberg-Marquardt-Kaczmarz iterative methods for solving systems of nonlinear ill-posed equations. Inverse Problems & Imaging, 2010, 4 (3) : 335-350. doi: 10.3934/ipi.2010.4.335
[5]

Zhili Ge, Gang Qian, Deren Han. Global convergence of an inexact operator splitting method for monotone variational inequalities. Journal of Industrial & Management Optimization, 2011, 7 (4) : 1013-1026. doi: 10.3934/jimo.2011.7.1013
[6]

Wei-Zhe Gu, Li-Yong Lu. The linear convergence of a derivative-free descent method for nonlinear complementarity problems. Journal of Industrial & Management Optimization, 2017, 13 (2) : 531-548. doi: 10.3934/jimo.2016030
[7]

Chunlin Hao, Xinwei Liu. Global convergence of an SQP algorithm for nonlinear optimization with overdetermined constraints. Numerical Algebra, Control & Optimization, 2012, 2 (1) : 19-29. doi: 10.3934/naco.2012.2.19
[8]

Zhong Tan, Qiuju Xu, Huaqiao Wang. Global existence and convergence rates for the compressible magnetohydrodynamic equations without heat conductivity. Discrete & Continuous Dynamical Systems - A, 2015, 35 (10) : 5083-5105. doi: 10.3934/dcds.2015.35.5083
[9]

Nora Merabet. Global convergence of a memory gradient method with closed-form step size formula. Conference Publications, 2007, 2007 (Special) : 721-730. doi: 10.3934/proc.2007.2007.721
[10]

Zhiyou Wu, Fusheng Bai, Guoquan Li, Yongjian Yang. A new auxiliary function method for systems of nonlinear equations. Journal of Industrial & Management Optimization, 2015, 11 (2) : 345-364. doi: 10.3934/jimo.2015.11.345
[11]

Martin Burger, José A. Carrillo, Marie-Therese Wolfram. A mixed finite element method for nonlinear diffusion equations. Kinetic & Related Models, 2010, 3 (1) : 59-83. doi: 10.3934/krm.2010.3.59
[12]

Thierry Colin, Boniface Nkonga. Multiscale numerical method for nonlinear Maxwell equations. Discrete & Continuous Dynamical Systems - B, 2005, 5 (3) : 631-658. doi: 10.3934/dcdsb.2005.5.631
[13]

Carlos E. Kenig. The method of energy channels for nonlinear wave equations. Discrete & Continuous Dynamical Systems - A, 2019, 39 (12) : 6979-6993. doi: 10.3934/dcds.2019240
[14]

Walter Allegretto, Yanping Lin, Zhiyong Zhang. Convergence to convection-diffusion waves for solutions to dissipative nonlinear evolution equations. Conference Publications, 2009, 2009 (Special) : 11-23. doi: 10.3934/proc.2009.2009.11
[15]

José A. Carrillo, Jean Dolbeault, Ivan Gentil, Ansgar Jüngel. Entropy-energy inequalities and improved convergence rates for nonlinear parabolic equations. Discrete & Continuous Dynamical Systems - B, 2006, 6 (5) : 1027-1050. doi: 10.3934/dcdsb.2006.6.1027
[16]

Yang Li, Yonghong Ren, Yun Wang, Jian Gu. Convergence analysis of a nonlinear Lagrangian method for nonconvex semidefinite programming with subproblem inexactly solved. Journal of Industrial & Management Optimization, 2015, 11 (1) : 65-81. doi: 10.3934/jimo.2015.11.65
[17]

W. Wei, H. M. Yin. Global solvability for a singular nonlinear Maxwell's equations. Communications on Pure & Applied Analysis, 2005, 4 (2) : 431-444. doi: 10.3934/cpaa.2005.4.431
[18]

Daniela Giachetti, Maria Michaela Porzio. Global existence for nonlinear parabolic equations with a damping term. Communications on Pure & Applied Analysis, 2009, 8 (3) : 923-953. doi: 10.3934/cpaa.2009.8.923
[19]

Monica Conti, Elsa M. Marchini, V. Pata. Global attractors for nonlinear viscoelastic equations with memory. Communications on Pure & Applied Analysis, 2016, 15 (5) : 1893-1913. doi: 10.3934/cpaa.2016021
[20]

Zuzana Došlá, Mauro Marini, Serena Matucci. Global Kneser solutions to nonlinear equations with indefinite weight. Discrete & Continuous Dynamical Systems - B, 2018, 23 (8) : 3297-3308. doi: 10.3934/dcdsb.2018252

 Impact Factor: 

Tools

Metrics

  • PDF downloads (9)
  • HTML views (0)
  • Cited by (0)

Other articles
by authors

[Back to Top]

Copyright © 2019 American Institute of Mathematical Sciences