American Institute of Mathematical Sciences

Advanced
  • Previous Article
    Analysis of complexity of primal-dual interior-point algorithms based on a new kernel function for linear optimization
  • NACO Home
  • This Issue
  • Next Article
    Delay-range dependent $H_\infty$ control for uncertain 2D-delayed systems
2015, 5(1): 25-36. doi: 10.3934/naco.2015.5.25

On the global convergence of a parameter-adjusting Levenberg-Marquardt method

Liyan Qi 1, , Xiantao Xiao 1, and  Liwei Zhang 1,

1. 

School of Mathematical Sciences, Dalian University of Technology, Dalian 116023, China, China

Received  January 2015 Revised  March 2015 Published  March 2015

The Levenberg-Marquardt (LM) method is a classical but popular method for solving nonlinear equations. Based on the trust region technique, we propose a parameter-adjusting LM (PALM) method, in which the LM parameter $\mu_k$ is self-adjusted at each iteration based on the ratio between actual reduction and predicted reduction. Under the level-bounded condition, we prove the global convergence of PALM. We also propose a modified parameter-adjusting LM (MPALM) method. Numerical results show that the two methods are very efficient.
Keywords: Nonlinear equations, global convergence., Levenberg-Marquardt method.
Mathematics Subject Classification: Primary: 90C30, 65K0.
Citation: Liyan Qi, Xiantao Xiao, Liwei Zhang. On the global convergence of a parameter-adjusting Levenberg-Marquardt method. Numerical Algebra, Control and Optimization, 2015, 5 (1) : 25-36. doi: 10.3934/naco.2015.5.25
References:
[1]

J. Fan, The modified Levenberg-Marquardt method for nonlinear equations with cubic convergence, Math. Comp., 81 (2012), 447-466. doi: 10.1090/S0025-5718-2011-02496-8.

[2]

J. Fan, Accelerating the modified Levenberg-Marquardt method for nonlinear equations, Math. Comp., 83 (2014), 1173-1187. doi: 10.1090/S0025-5718-2013-02752-4.

[3]

J. Fan and J. Pan, Convergence properties of a self-adaptive Levenberg-Marquardt algorithm under local error bound condition, Comput. Optim. Appl., 34 (2006), 47-62. doi: 10.1007/s10589-005-3074-z.

[4]

J. Fan and Y. Yuan, On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption, Computing, 74 (2005), 23-39. doi: 10.1007/s00607-004-0083-1.

[5]

K. Levenberg, A method for the solution of certain nonlinear problems in least squares, Quart. Appl. Math., 2 (1944), 164-166.

[6]

D. W. Marquardt, An algorithm for least-squares estimation of nonlinear inequalities, SIAM J. Appl. Math., 11 (1963), 431-441.

[7]

J. J. Moré, Recent developments in algorithms and software for trust region methods, in Mathematical Programming: the state of the art (Bonn, 1982), Springer, Berlin, (1983), 258-287.

[8]

J. J. Moré, B. S. Garbow and K. E. Hillstrom, Testing unconstrained optimization software, ACM Trans. Math. Software, 7 (1981), 17-41. doi: 10.1145/355934.355936.

[9]

J. Nocedal and S. J. Wright, Numerical optimization, 2nd edition, Springer Series in Operations Research and Financial Engineering, Springer, New York, 2006.

[10]

N. Yamashita and M. Fukushima, On the rate of convergence of the Levenberg-Marquardt method, in Topics in numerical analysis, Comput. Suppl., Springer, Vienna, 15 (2001), 239-249. doi: 10.1007/978-3-7091-6217-0_18.

show all references

References:
[1]

J. Fan, The modified Levenberg-Marquardt method for nonlinear equations with cubic convergence, Math. Comp., 81 (2012), 447-466. doi: 10.1090/S0025-5718-2011-02496-8.

[2]

J. Fan, Accelerating the modified Levenberg-Marquardt method for nonlinear equations, Math. Comp., 83 (2014), 1173-1187. doi: 10.1090/S0025-5718-2013-02752-4.

[3]

J. Fan and J. Pan, Convergence properties of a self-adaptive Levenberg-Marquardt algorithm under local error bound condition, Comput. Optim. Appl., 34 (2006), 47-62. doi: 10.1007/s10589-005-3074-z.

[4]

J. Fan and Y. Yuan, On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption, Computing, 74 (2005), 23-39. doi: 10.1007/s00607-004-0083-1.

[5]

K. Levenberg, A method for the solution of certain nonlinear problems in least squares, Quart. Appl. Math., 2 (1944), 164-166.

[6]

D. W. Marquardt, An algorithm for least-squares estimation of nonlinear inequalities, SIAM J. Appl. Math., 11 (1963), 431-441.

[7]

J. J. Moré, Recent developments in algorithms and software for trust region methods, in Mathematical Programming: the state of the art (Bonn, 1982), Springer, Berlin, (1983), 258-287.

[8]

J. J. Moré, B. S. Garbow and K. E. Hillstrom, Testing unconstrained optimization software, ACM Trans. Math. Software, 7 (1981), 17-41. doi: 10.1145/355934.355936.

[9]

J. Nocedal and S. J. Wright, Numerical optimization, 2nd edition, Springer Series in Operations Research and Financial Engineering, Springer, New York, 2006.

[10]

N. Yamashita and M. Fukushima, On the rate of convergence of the Levenberg-Marquardt method, in Topics in numerical analysis, Comput. Suppl., Springer, Vienna, 15 (2001), 239-249. doi: 10.1007/978-3-7091-6217-0_18.

[1]

Jinyan Fan, Jianyu Pan. Inexact Levenberg-Marquardt method for nonlinear equations. Discrete and Continuous Dynamical Systems - B, 2004, 4 (4) : 1223-1232. doi: 10.3934/dcdsb.2004.4.1223
[2]

Jinyan Fan, Jianyu Pan. On the convergence rate of the inexact Levenberg-Marquardt method. Journal of Industrial and Management Optimization, 2011, 7 (1) : 199-210. doi: 10.3934/jimo.2011.7.199
[3]

Jinyan Fan. On the Levenberg-Marquardt methods for convex constrained nonlinear equations. Journal of Industrial and Management Optimization, 2013, 9 (1) : 227-241. doi: 10.3934/jimo.2013.9.227
[4]

Haiyan Wang, Jinyan Fan. Convergence properties of inexact Levenberg-Marquardt method under Hölderian local error bound. Journal of Industrial and Management Optimization, 2021, 17 (4) : 2265-2275. doi: 10.3934/jimo.2020068
[5]

Xin-He Miao, Kai Yao, Ching-Yu Yang, Jein-Shan Chen. Levenberg-Marquardt method for absolute value equation associated with second-order cone. Numerical Algebra, Control and Optimization, 2022, 12 (1) : 47-61. doi: 10.3934/naco.2021050
[6]

Johann Baumeister, Barbara Kaltenbacher, Antonio Leitão. On Levenberg-Marquardt-Kaczmarz iterative methods for solving systems of nonlinear ill-posed equations. Inverse Problems and Imaging, 2010, 4 (3) : 335-350. doi: 10.3934/ipi.2010.4.335
[7]

Jiangxing Wang. Convergence analysis of an accurate and efficient method for nonlinear Maxwell's equations. Discrete and Continuous Dynamical Systems - B, 2021, 26 (5) : 2429-2440. doi: 10.3934/dcdsb.2020185
[8]

Jahnabi Chakravarty, Ashiho Athikho, Manideepa Saha. Convergence of interval AOR method for linear interval equations. Numerical Algebra, Control and Optimization, 2022, 12 (2) : 293-308. doi: 10.3934/naco.2021006
[9]

Zhili Ge, Gang Qian, Deren Han. Global convergence of an inexact operator splitting method for monotone variational inequalities. Journal of Industrial and Management Optimization, 2011, 7 (4) : 1013-1026. doi: 10.3934/jimo.2011.7.1013
[10]

Gonglin Yuan, Zhan Wang, Pengyuan Li. Global convergence of a modified Broyden family method for nonconvex functions. Journal of Industrial and Management Optimization, 2021  doi: 10.3934/jimo.2021164
[11]

Wei-Zhe Gu, Li-Yong Lu. The linear convergence of a derivative-free descent method for nonlinear complementarity problems. Journal of Industrial and Management Optimization, 2017, 13 (2) : 531-548. doi: 10.3934/jimo.2016030
[12]

Chunlin Hao, Xinwei Liu. Global convergence of an SQP algorithm for nonlinear optimization with overdetermined constraints. Numerical Algebra, Control and Optimization, 2012, 2 (1) : 19-29. doi: 10.3934/naco.2012.2.19
[13]

Zhong Tan, Qiuju Xu, Huaqiao Wang. Global existence and convergence rates for the compressible magnetohydrodynamic equations without heat conductivity. Discrete and Continuous Dynamical Systems, 2015, 35 (10) : 5083-5105. doi: 10.3934/dcds.2015.35.5083
[14]

Ruiying Wei, Yin Li, Zheng-an Yao. Global existence and convergence rates of solutions for the compressible Euler equations with damping. Discrete and Continuous Dynamical Systems - B, 2020, 25 (8) : 2949-2967. doi: 10.3934/dcdsb.2020047
[15]

Walter Allegretto, Yanping Lin, Zhiyong Zhang. Convergence to convection-diffusion waves for solutions to dissipative nonlinear evolution equations. Conference Publications, 2009, 2009 (Special) : 11-23. doi: 10.3934/proc.2009.2009.11
[16]

José A. Carrillo, Jean Dolbeault, Ivan Gentil, Ansgar Jüngel. Entropy-energy inequalities and improved convergence rates for nonlinear parabolic equations. Discrete and Continuous Dynamical Systems - B, 2006, 6 (5) : 1027-1050. doi: 10.3934/dcdsb.2006.6.1027
[17]

Nora Merabet. Global convergence of a memory gradient method with closed-form step size formula. Conference Publications, 2007, 2007 (Special) : 721-730. doi: 10.3934/proc.2007.2007.721
[18]

Zhiyou Wu, Fusheng Bai, Guoquan Li, Yongjian Yang. A new auxiliary function method for systems of nonlinear equations. Journal of Industrial and Management Optimization, 2015, 11 (2) : 345-364. doi: 10.3934/jimo.2015.11.345
[19]

Carlos E. Kenig. The method of energy channels for nonlinear wave equations. Discrete and Continuous Dynamical Systems, 2019, 39 (12) : 6979-6993. doi: 10.3934/dcds.2019240
[20]

Martin Burger, José A. Carrillo, Marie-Therese Wolfram. A mixed finite element method for nonlinear diffusion equations. Kinetic and Related Models, 2010, 3 (1) : 59-83. doi: 10.3934/krm.2010.3.59

 Impact Factor: 

Tools

Metrics

  • PDF downloads (165)
  • HTML views (0)
  • Cited by (1)

Other articles
by authors

[Back to Top]

Copyright © 2022 American Institute of Mathematical Sciences | Privacy Policy