• Previous Article
    Analysis of complexity of primal-dual interior-point algorithms based on a new kernel function for linear optimization
  • NACO Home
  • This Issue
  • Next Article
    Delay-range dependent $H_\infty$ control for uncertain 2D-delayed systems
2015, 5(1): 25-36. doi: 10.3934/naco.2015.5.25

On the global convergence of a parameter-adjusting Levenberg-Marquardt method

1. 

School of Mathematical Sciences, Dalian University of Technology, Dalian 116023, China, China

Received  January 2015 Revised  March 2015 Published  March 2015

The Levenberg-Marquardt (LM) method is a classical but popular method for solving nonlinear equations. Based on the trust region technique, we propose a parameter-adjusting LM (PALM) method, in which the LM parameter $\mu_k$ is self-adjusted at each iteration based on the ratio between actual reduction and predicted reduction. Under the level-bounded condition, we prove the global convergence of PALM. We also propose a modified parameter-adjusting LM (MPALM) method. Numerical results show that the two methods are very efficient.
Citation: Liyan Qi, Xiantao Xiao, Liwei Zhang. On the global convergence of a parameter-adjusting Levenberg-Marquardt method. Numerical Algebra, Control & Optimization, 2015, 5 (1) : 25-36. doi: 10.3934/naco.2015.5.25
References:
[1]

J. Fan, The modified Levenberg-Marquardt method for nonlinear equations with cubic convergence,, Math. Comp., 81 (2012), 447.  doi: 10.1090/S0025-5718-2011-02496-8.  Google Scholar

[2]

J. Fan, Accelerating the modified Levenberg-Marquardt method for nonlinear equations,, Math. Comp., 83 (2014), 1173.  doi: 10.1090/S0025-5718-2013-02752-4.  Google Scholar

[3]

J. Fan and J. Pan, Convergence properties of a self-adaptive Levenberg-Marquardt algorithm under local error bound condition,, Comput. Optim. Appl., 34 (2006), 47.  doi: 10.1007/s10589-005-3074-z.  Google Scholar

[4]

J. Fan and Y. Yuan, On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption,, Computing, 74 (2005), 23.  doi: 10.1007/s00607-004-0083-1.  Google Scholar

[5]

K. Levenberg, A method for the solution of certain nonlinear problems in least squares,, Quart. Appl. Math., 2 (1944), 164.   Google Scholar

[6]

D. W. Marquardt, An algorithm for least-squares estimation of nonlinear inequalities,, SIAM J. Appl. Math., 11 (1963), 431.   Google Scholar

[7]

J. J. Moré, Recent developments in algorithms and software for trust region methods,, in Mathematical Programming: the state of the art (Bonn, (1983), 258.   Google Scholar

[8]

J. J. Moré, B. S. Garbow and K. E. Hillstrom, Testing unconstrained optimization software,, ACM Trans. Math. Software, 7 (1981), 17.  doi: 10.1145/355934.355936.  Google Scholar

[9]

J. Nocedal and S. J. Wright, Numerical optimization,, 2nd edition, (2006).   Google Scholar

[10]

N. Yamashita and M. Fukushima, On the rate of convergence of the Levenberg-Marquardt method,, in Topics in numerical analysis, 15 (2001), 239.  doi: 10.1007/978-3-7091-6217-0_18.  Google Scholar

show all references

References:
[1]

J. Fan, The modified Levenberg-Marquardt method for nonlinear equations with cubic convergence,, Math. Comp., 81 (2012), 447.  doi: 10.1090/S0025-5718-2011-02496-8.  Google Scholar

[2]

J. Fan, Accelerating the modified Levenberg-Marquardt method for nonlinear equations,, Math. Comp., 83 (2014), 1173.  doi: 10.1090/S0025-5718-2013-02752-4.  Google Scholar

[3]

J. Fan and J. Pan, Convergence properties of a self-adaptive Levenberg-Marquardt algorithm under local error bound condition,, Comput. Optim. Appl., 34 (2006), 47.  doi: 10.1007/s10589-005-3074-z.  Google Scholar

[4]

J. Fan and Y. Yuan, On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption,, Computing, 74 (2005), 23.  doi: 10.1007/s00607-004-0083-1.  Google Scholar

[5]

K. Levenberg, A method for the solution of certain nonlinear problems in least squares,, Quart. Appl. Math., 2 (1944), 164.   Google Scholar

[6]

D. W. Marquardt, An algorithm for least-squares estimation of nonlinear inequalities,, SIAM J. Appl. Math., 11 (1963), 431.   Google Scholar

[7]

J. J. Moré, Recent developments in algorithms and software for trust region methods,, in Mathematical Programming: the state of the art (Bonn, (1983), 258.   Google Scholar

[8]

J. J. Moré, B. S. Garbow and K. E. Hillstrom, Testing unconstrained optimization software,, ACM Trans. Math. Software, 7 (1981), 17.  doi: 10.1145/355934.355936.  Google Scholar

[9]

J. Nocedal and S. J. Wright, Numerical optimization,, 2nd edition, (2006).   Google Scholar

[10]

N. Yamashita and M. Fukushima, On the rate of convergence of the Levenberg-Marquardt method,, in Topics in numerical analysis, 15 (2001), 239.  doi: 10.1007/978-3-7091-6217-0_18.  Google Scholar

[1]

Zuliang Lu, Fei Huang, Xiankui Wu, Lin Li, Shang Liu. Convergence and quasi-optimality of $ L^2- $norms based an adaptive finite element method for nonlinear optimal control problems. Electronic Research Archive, 2020, 28 (4) : 1459-1486. doi: 10.3934/era.2020077

[2]

Hassan Mohammad. A diagonal PRP-type projection method for convex constrained nonlinear monotone equations. Journal of Industrial & Management Optimization, 2021, 17 (1) : 101-116. doi: 10.3934/jimo.2019101

[3]

Gang Luo, Qingzhi Yang. The point-wise convergence of shifted symmetric higher order power method. Journal of Industrial & Management Optimization, 2021, 17 (1) : 357-368. doi: 10.3934/jimo.2019115

[4]

Bing Yu, Lei Zhang. Global optimization-based dimer method for finding saddle points. Discrete & Continuous Dynamical Systems - B, 2021, 26 (1) : 741-753. doi: 10.3934/dcdsb.2020139

[5]

Gunther Uhlmann, Jian Zhai. Inverse problems for nonlinear hyperbolic equations. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 455-469. doi: 10.3934/dcds.2020380

[6]

Yuxia Guo, Shaolong Peng. A direct method of moving planes for fully nonlinear nonlocal operators and applications. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020462

[7]

Xiu Ye, Shangyou Zhang, Peng Zhu. A weak Galerkin finite element method for nonlinear conservation laws. Electronic Research Archive, 2021, 29 (1) : 1897-1923. doi: 10.3934/era.2020097

[8]

Wenjun Liu, Hefeng Zhuang. Global attractor for a suspension bridge problem with a nonlinear delay term in the internal feedback. Discrete & Continuous Dynamical Systems - B, 2021, 26 (2) : 907-942. doi: 10.3934/dcdsb.2020147

[9]

Ke Su, Yumeng Lin, Chun Xu. A new adaptive method to nonlinear semi-infinite programming. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2021012

[10]

Ziang Long, Penghang Yin, Jack Xin. Global convergence and geometric characterization of slow to fast weight evolution in neural network training for classifying linearly non-separable data. Inverse Problems & Imaging, 2021, 15 (1) : 41-62. doi: 10.3934/ipi.2020077

[11]

Thomas Bartsch, Tian Xu. Strongly localized semiclassical states for nonlinear Dirac equations. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 29-60. doi: 10.3934/dcds.2020297

[12]

Hua Chen, Yawei Wei. Multiple solutions for nonlinear cone degenerate elliptic equations. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2020272

[13]

Matthieu Alfaro, Isabeau Birindelli. Evolution equations involving nonlinear truncated Laplacian operators. Discrete & Continuous Dynamical Systems - A, 2020, 40 (6) : 3057-3073. doi: 10.3934/dcds.2020046

[14]

Li-Bin Liu, Ying Liang, Jian Zhang, Xiaobing Bao. A robust adaptive grid method for singularly perturbed Burger-Huxley equations. Electronic Research Archive, 2020, 28 (4) : 1439-1457. doi: 10.3934/era.2020076

[15]

Karoline Disser. Global existence and uniqueness for a volume-surface reaction-nonlinear-diffusion system. Discrete & Continuous Dynamical Systems - S, 2021, 14 (1) : 321-330. doi: 10.3934/dcdss.2020326

[16]

Scipio Cuccagna, Masaya Maeda. A survey on asymptotic stability of ground states of nonlinear Schrödinger equations II. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020450

[17]

Junyong Eom, Kazuhiro Ishige. Large time behavior of ODE type solutions to nonlinear diffusion equations. Discrete & Continuous Dynamical Systems - A, 2020, 40 (6) : 3395-3409. doi: 10.3934/dcds.2019229

[18]

Nguyen Huu Can, Nguyen Huy Tuan, Donal O'Regan, Vo Van Au. On a final value problem for a class of nonlinear hyperbolic equations with damping term. Evolution Equations & Control Theory, 2021, 10 (1) : 103-127. doi: 10.3934/eect.2020053

[19]

Andrew Comech, Scipio Cuccagna. On asymptotic stability of ground states of some systems of nonlinear Schrödinger equations. Discrete & Continuous Dynamical Systems - A, 2021, 41 (3) : 1225-1270. doi: 10.3934/dcds.2020316

[20]

Marion Darbas, Jérémy Heleine, Stephanie Lohrengel. Numerical resolution by the quasi-reversibility method of a data completion problem for Maxwell's equations. Inverse Problems & Imaging, 2020, 14 (6) : 1107-1133. doi: 10.3934/ipi.2020056

 Impact Factor: 

Metrics

  • PDF downloads (58)
  • HTML views (0)
  • Cited by (1)

Other articles
by authors

[Back to Top]