American Institute of Mathematical Sciences

Advanced
  • Previous Article
    Analysis of complexity of primal-dual interior-point algorithms based on a new kernel function for linear optimization
  • NACO Home
  • This Issue
  • Next Article
    Delay-range dependent $H_\infty$ control for uncertain 2D-delayed systems
2015, 5(1): 25-36. doi: 10.3934/naco.2015.5.25

On the global convergence of a parameter-adjusting Levenberg-Marquardt method

Liyan Qi 1, , Xiantao Xiao 1, and  Liwei Zhang 1,

1. 

School of Mathematical Sciences, Dalian University of Technology, Dalian 116023, China, China

Received  January 2015 Revised  March 2015 Published  March 2015

The Levenberg-Marquardt (LM) method is a classical but popular method for solving nonlinear equations. Based on the trust region technique, we propose a parameter-adjusting LM (PALM) method, in which the LM parameter $\mu_k$ is self-adjusted at each iteration based on the ratio between actual reduction and predicted reduction. Under the level-bounded condition, we prove the global convergence of PALM. We also propose a modified parameter-adjusting LM (MPALM) method. Numerical results show that the two methods are very efficient.
Keywords: Nonlinear equations, global convergence., Levenberg-Marquardt method.
Mathematics Subject Classification: Primary: 90C30, 65K0.
Citation: Liyan Qi, Xiantao Xiao, Liwei Zhang. On the global convergence of a parameter-adjusting Levenberg-Marquardt method. Numerical Algebra, Control & Optimization, 2015, 5 (1) : 25-36. doi: 10.3934/naco.2015.5.25
References:
[1]

J. Fan, The modified Levenberg-Marquardt method for nonlinear equations with cubic convergence, Math. Comp., 81 (2012), 447-466. doi: 10.1090/S0025-5718-2011-02496-8.  Google Scholar

[2]

J. Fan, Accelerating the modified Levenberg-Marquardt method for nonlinear equations, Math. Comp., 83 (2014), 1173-1187. doi: 10.1090/S0025-5718-2013-02752-4.  Google Scholar

[3]

J. Fan and J. Pan, Convergence properties of a self-adaptive Levenberg-Marquardt algorithm under local error bound condition, Comput. Optim. Appl., 34 (2006), 47-62. doi: 10.1007/s10589-005-3074-z.  Google Scholar

[4]

J. Fan and Y. Yuan, On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption, Computing, 74 (2005), 23-39. doi: 10.1007/s00607-004-0083-1.  Google Scholar

[5]

K. Levenberg, A method for the solution of certain nonlinear problems in least squares, Quart. Appl. Math., 2 (1944), 164-166.  Google Scholar

[6]

D. W. Marquardt, An algorithm for least-squares estimation of nonlinear inequalities, SIAM J. Appl. Math., 11 (1963), 431-441.  Google Scholar

[7]

J. J. Moré, Recent developments in algorithms and software for trust region methods, in Mathematical Programming: the state of the art (Bonn, 1982), Springer, Berlin, (1983), 258-287.  Google Scholar

[8]

J. J. Moré, B. S. Garbow and K. E. Hillstrom, Testing unconstrained optimization software, ACM Trans. Math. Software, 7 (1981), 17-41. doi: 10.1145/355934.355936.  Google Scholar

[9]

J. Nocedal and S. J. Wright, Numerical optimization, 2nd edition, Springer Series in Operations Research and Financial Engineering, Springer, New York, 2006.  Google Scholar

[10]

N. Yamashita and M. Fukushima, On the rate of convergence of the Levenberg-Marquardt method, in Topics in numerical analysis, Comput. Suppl., Springer, Vienna, 15 (2001), 239-249. doi: 10.1007/978-3-7091-6217-0_18.  Google Scholar

show all references

References:
[1]

J. Fan, The modified Levenberg-Marquardt method for nonlinear equations with cubic convergence, Math. Comp., 81 (2012), 447-466. doi: 10.1090/S0025-5718-2011-02496-8.  Google Scholar

[2]

J. Fan, Accelerating the modified Levenberg-Marquardt method for nonlinear equations, Math. Comp., 83 (2014), 1173-1187. doi: 10.1090/S0025-5718-2013-02752-4.  Google Scholar

[3]

J. Fan and J. Pan, Convergence properties of a self-adaptive Levenberg-Marquardt algorithm under local error bound condition, Comput. Optim. Appl., 34 (2006), 47-62. doi: 10.1007/s10589-005-3074-z.  Google Scholar

[4]

J. Fan and Y. Yuan, On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption, Computing, 74 (2005), 23-39. doi: 10.1007/s00607-004-0083-1.  Google Scholar

[5]

K. Levenberg, A method for the solution of certain nonlinear problems in least squares, Quart. Appl. Math., 2 (1944), 164-166.  Google Scholar

[6]

D. W. Marquardt, An algorithm for least-squares estimation of nonlinear inequalities, SIAM J. Appl. Math., 11 (1963), 431-441.  Google Scholar

[7]

J. J. Moré, Recent developments in algorithms and software for trust region methods, in Mathematical Programming: the state of the art (Bonn, 1982), Springer, Berlin, (1983), 258-287.  Google Scholar

[8]

J. J. Moré, B. S. Garbow and K. E. Hillstrom, Testing unconstrained optimization software, ACM Trans. Math. Software, 7 (1981), 17-41. doi: 10.1145/355934.355936.  Google Scholar

[9]

J. Nocedal and S. J. Wright, Numerical optimization, 2nd edition, Springer Series in Operations Research and Financial Engineering, Springer, New York, 2006.  Google Scholar

[10]

N. Yamashita and M. Fukushima, On the rate of convergence of the Levenberg-Marquardt method, in Topics in numerical analysis, Comput. Suppl., Springer, Vienna, 15 (2001), 239-249. doi: 10.1007/978-3-7091-6217-0_18.  Google Scholar

[1]

Jinyan Fan, Jianyu Pan. Inexact Levenberg-Marquardt method for nonlinear equations. Discrete & Continuous Dynamical Systems - B, 2004, 4 (4) : 1223-1232. doi: 10.3934/dcdsb.2004.4.1223
[2]

Jinyan Fan, Jianyu Pan. On the convergence rate of the inexact Levenberg-Marquardt method. Journal of Industrial & Management Optimization, 2011, 7 (1) : 199-210. doi: 10.3934/jimo.2011.7.199
[3]

Jinyan Fan. On the Levenberg-Marquardt methods for convex constrained nonlinear equations. Journal of Industrial & Management Optimization, 2013, 9 (1) : 227-241. doi: 10.3934/jimo.2013.9.227
[4]

Haiyan Wang, Jinyan Fan. Convergence properties of inexact Levenberg-Marquardt method under Hölderian local error bound. Journal of Industrial & Management Optimization, 2021, 17 (4) : 2265-2275. doi: 10.3934/jimo.2020068
[5]

Johann Baumeister, Barbara Kaltenbacher, Antonio Leitão. On Levenberg-Marquardt-Kaczmarz iterative methods for solving systems of nonlinear ill-posed equations. Inverse Problems & Imaging, 2010, 4 (3) : 335-350. doi: 10.3934/ipi.2010.4.335
[6]

Jiangxing Wang. Convergence analysis of an accurate and efficient method for nonlinear Maxwell's equations. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2429-2440. doi: 10.3934/dcdsb.2020185
[7]

Jahnabi Chakravarty, Ashiho Athikho, Manideepa Saha. Convergence of interval AOR method for linear interval equations. Numerical Algebra, Control & Optimization, 2021  doi: 10.3934/naco.2021006
[8]

Zhili Ge, Gang Qian, Deren Han. Global convergence of an inexact operator splitting method for monotone variational inequalities. Journal of Industrial & Management Optimization, 2011, 7 (4) : 1013-1026. doi: 10.3934/jimo.2011.7.1013
[9]

Gonglin Yuan, Zhan Wang, Pengyuan Li. Global convergence of a modified Broyden family method for nonconvex functions. Journal of Industrial & Management Optimization, 2021  doi: 10.3934/jimo.2021164
[10]

Wei-Zhe Gu, Li-Yong Lu. The linear convergence of a derivative-free descent method for nonlinear complementarity problems. Journal of Industrial & Management Optimization, 2017, 13 (2) : 531-548. doi: 10.3934/jimo.2016030
[11]

Chunlin Hao, Xinwei Liu. Global convergence of an SQP algorithm for nonlinear optimization with overdetermined constraints. Numerical Algebra, Control & Optimization, 2012, 2 (1) : 19-29. doi: 10.3934/naco.2012.2.19
[12]

Zhong Tan, Qiuju Xu, Huaqiao Wang. Global existence and convergence rates for the compressible magnetohydrodynamic equations without heat conductivity. Discrete & Continuous Dynamical Systems, 2015, 35 (10) : 5083-5105. doi: 10.3934/dcds.2015.35.5083
[13]

Ruiying Wei, Yin Li, Zheng-an Yao. Global existence and convergence rates of solutions for the compressible Euler equations with damping. Discrete & Continuous Dynamical Systems - B, 2020, 25 (8) : 2949-2967. doi: 10.3934/dcdsb.2020047
[14]

Walter Allegretto, Yanping Lin, Zhiyong Zhang. Convergence to convection-diffusion waves for solutions to dissipative nonlinear evolution equations. Conference Publications, 2009, 2009 (Special) : 11-23. doi: 10.3934/proc.2009.2009.11
[15]

José A. Carrillo, Jean Dolbeault, Ivan Gentil, Ansgar Jüngel. Entropy-energy inequalities and improved convergence rates for nonlinear parabolic equations. Discrete & Continuous Dynamical Systems - B, 2006, 6 (5) : 1027-1050. doi: 10.3934/dcdsb.2006.6.1027
[16]

Nora Merabet. Global convergence of a memory gradient method with closed-form step size formula. Conference Publications, 2007, 2007 (Special) : 721-730. doi: 10.3934/proc.2007.2007.721
[17]

Zhiyou Wu, Fusheng Bai, Guoquan Li, Yongjian Yang. A new auxiliary function method for systems of nonlinear equations. Journal of Industrial & Management Optimization, 2015, 11 (2) : 345-364. doi: 10.3934/jimo.2015.11.345
[18]

Carlos E. Kenig. The method of energy channels for nonlinear wave equations. Discrete & Continuous Dynamical Systems, 2019, 39 (12) : 6979-6993. doi: 10.3934/dcds.2019240
[19]

Weijun Zhou. A globally convergent BFGS method for symmetric nonlinear equations. Journal of Industrial & Management Optimization, 2021  doi: 10.3934/jimo.2021020
[20]

Martin Burger, José A. Carrillo, Marie-Therese Wolfram. A mixed finite element method for nonlinear diffusion equations. Kinetic & Related Models, 2010, 3 (1) : 59-83. doi: 10.3934/krm.2010.3.59

 Impact Factor: 

Tools

Metrics

  • PDF downloads (103)
  • HTML views (0)
  • Cited by (1)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences