American Institute of Mathematical Sciences

Advanced
  • Previous Article
    Analysis of complexity of primal-dual interior-point algorithms based on a new kernel function for linear optimization
  • NACO Home
  • This Issue
  • Next Article
    Delay-range dependent $H_\infty$ control for uncertain 2D-delayed systems
2015, 5(1): 25-36. doi: 10.3934/naco.2015.5.25

On the global convergence of a parameter-adjusting Levenberg-Marquardt method

Liyan Qi 1, , Xiantao Xiao 1, and  Liwei Zhang 1,

1. 

School of Mathematical Sciences, Dalian University of Technology, Dalian 116023, China, China

Received  January 2015 Revised  March 2015 Published  March 2015

The Levenberg-Marquardt (LM) method is a classical but popular method for solving nonlinear equations. Based on the trust region technique, we propose a parameter-adjusting LM (PALM) method, in which the LM parameter $\mu_k$ is self-adjusted at each iteration based on the ratio between actual reduction and predicted reduction. Under the level-bounded condition, we prove the global convergence of PALM. We also propose a modified parameter-adjusting LM (MPALM) method. Numerical results show that the two methods are very efficient.
Keywords: Nonlinear equations, global convergence., Levenberg-Marquardt method.
Mathematics Subject Classification: Primary: 90C30, 65K0.
Citation: Liyan Qi, Xiantao Xiao, Liwei Zhang. On the global convergence of a parameter-adjusting Levenberg-Marquardt method. Numerical Algebra, Control & Optimization, 2015, 5 (1) : 25-36. doi: 10.3934/naco.2015.5.25
References:
[1]

J. Fan, The modified Levenberg-Marquardt method for nonlinear equations with cubic convergence,, Math. Comp., 81 (2012), 447.  doi: 10.1090/S0025-5718-2011-02496-8.  Google Scholar

[2]

J. Fan, Accelerating the modified Levenberg-Marquardt method for nonlinear equations,, Math. Comp., 83 (2014), 1173.  doi: 10.1090/S0025-5718-2013-02752-4.  Google Scholar

[3]

J. Fan and J. Pan, Convergence properties of a self-adaptive Levenberg-Marquardt algorithm under local error bound condition,, Comput. Optim. Appl., 34 (2006), 47.  doi: 10.1007/s10589-005-3074-z.  Google Scholar

[4]

J. Fan and Y. Yuan, On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption,, Computing, 74 (2005), 23.  doi: 10.1007/s00607-004-0083-1.  Google Scholar

[5]

K. Levenberg, A method for the solution of certain nonlinear problems in least squares,, Quart. Appl. Math., 2 (1944), 164.   Google Scholar

[6]

D. W. Marquardt, An algorithm for least-squares estimation of nonlinear inequalities,, SIAM J. Appl. Math., 11 (1963), 431.   Google Scholar

[7]

J. J. Moré, Recent developments in algorithms and software for trust region methods,, in Mathematical Programming: the state of the art (Bonn, (1983), 258.   Google Scholar

[8]

J. J. Moré, B. S. Garbow and K. E. Hillstrom, Testing unconstrained optimization software,, ACM Trans. Math. Software, 7 (1981), 17.  doi: 10.1145/355934.355936.  Google Scholar

[9]

J. Nocedal and S. J. Wright, Numerical optimization,, 2nd edition, (2006).   Google Scholar

[10]

N. Yamashita and M. Fukushima, On the rate of convergence of the Levenberg-Marquardt method,, in Topics in numerical analysis, 15 (2001), 239.  doi: 10.1007/978-3-7091-6217-0_18.  Google Scholar

show all references

References:
[1]

J. Fan, The modified Levenberg-Marquardt method for nonlinear equations with cubic convergence,, Math. Comp., 81 (2012), 447.  doi: 10.1090/S0025-5718-2011-02496-8.  Google Scholar

[2]

J. Fan, Accelerating the modified Levenberg-Marquardt method for nonlinear equations,, Math. Comp., 83 (2014), 1173.  doi: 10.1090/S0025-5718-2013-02752-4.  Google Scholar

[3]

J. Fan and J. Pan, Convergence properties of a self-adaptive Levenberg-Marquardt algorithm under local error bound condition,, Comput. Optim. Appl., 34 (2006), 47.  doi: 10.1007/s10589-005-3074-z.  Google Scholar

[4]

J. Fan and Y. Yuan, On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption,, Computing, 74 (2005), 23.  doi: 10.1007/s00607-004-0083-1.  Google Scholar

[5]

K. Levenberg, A method for the solution of certain nonlinear problems in least squares,, Quart. Appl. Math., 2 (1944), 164.   Google Scholar

[6]

D. W. Marquardt, An algorithm for least-squares estimation of nonlinear inequalities,, SIAM J. Appl. Math., 11 (1963), 431.   Google Scholar

[7]

J. J. Moré, Recent developments in algorithms and software for trust region methods,, in Mathematical Programming: the state of the art (Bonn, (1983), 258.   Google Scholar

[8]

J. J. Moré, B. S. Garbow and K. E. Hillstrom, Testing unconstrained optimization software,, ACM Trans. Math. Software, 7 (1981), 17.  doi: 10.1145/355934.355936.  Google Scholar

[9]

J. Nocedal and S. J. Wright, Numerical optimization,, 2nd edition, (2006).   Google Scholar

[10]

N. Yamashita and M. Fukushima, On the rate of convergence of the Levenberg-Marquardt method,, in Topics in numerical analysis, 15 (2001), 239.  doi: 10.1007/978-3-7091-6217-0_18.  Google Scholar

[1]

Zuliang Lu, Fei Huang, Xiankui Wu, Lin Li, Shang Liu. Convergence and quasi-optimality of $ L^2- $norms based an adaptive finite element method for nonlinear optimal control problems. Electronic Research Archive, 2020, 28 (4) : 1459-1486. doi: 10.3934/era.2020077
[2]

Hassan Mohammad. A diagonal PRP-type projection method for convex constrained nonlinear monotone equations. Journal of Industrial & Management Optimization, 2021, 17 (1) : 101-116. doi: 10.3934/jimo.2019101
[3]

Gang Luo, Qingzhi Yang. The point-wise convergence of shifted symmetric higher order power method. Journal of Industrial & Management Optimization, 2021, 17 (1) : 357-368. doi: 10.3934/jimo.2019115
[4]

Bing Yu, Lei Zhang. Global optimization-based dimer method for finding saddle points. Discrete & Continuous Dynamical Systems - B, 2021, 26 (1) : 741-753. doi: 10.3934/dcdsb.2020139
[5]

Gunther Uhlmann, Jian Zhai. Inverse problems for nonlinear hyperbolic equations. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 455-469. doi: 10.3934/dcds.2020380
[6]

Yuxia Guo, Shaolong Peng. A direct method of moving planes for fully nonlinear nonlocal operators and applications. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020462
[7]

Xiu Ye, Shangyou Zhang, Peng Zhu. A weak Galerkin finite element method for nonlinear conservation laws. Electronic Research Archive, 2021, 29 (1) : 1897-1923. doi: 10.3934/era.2020097
[8]

Wenjun Liu, Hefeng Zhuang. Global attractor for a suspension bridge problem with a nonlinear delay term in the internal feedback. Discrete & Continuous Dynamical Systems - B, 2021, 26 (2) : 907-942. doi: 10.3934/dcdsb.2020147
[9]

Ke Su, Yumeng Lin, Chun Xu. A new adaptive method to nonlinear semi-infinite programming. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2021012
[10]

Ziang Long, Penghang Yin, Jack Xin. Global convergence and geometric characterization of slow to fast weight evolution in neural network training for classifying linearly non-separable data. Inverse Problems & Imaging, 2021, 15 (1) : 41-62. doi: 10.3934/ipi.2020077
[11]

Thomas Bartsch, Tian Xu. Strongly localized semiclassical states for nonlinear Dirac equations. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 29-60. doi: 10.3934/dcds.2020297
[12]

Hua Chen, Yawei Wei. Multiple solutions for nonlinear cone degenerate elliptic equations. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2020272
[13]

Matthieu Alfaro, Isabeau Birindelli. Evolution equations involving nonlinear truncated Laplacian operators. Discrete & Continuous Dynamical Systems - A, 2020, 40 (6) : 3057-3073. doi: 10.3934/dcds.2020046
[14]

Li-Bin Liu, Ying Liang, Jian Zhang, Xiaobing Bao. A robust adaptive grid method for singularly perturbed Burger-Huxley equations. Electronic Research Archive, 2020, 28 (4) : 1439-1457. doi: 10.3934/era.2020076
[15]

Karoline Disser. Global existence and uniqueness for a volume-surface reaction-nonlinear-diffusion system. Discrete & Continuous Dynamical Systems - S, 2021, 14 (1) : 321-330. doi: 10.3934/dcdss.2020326
[16]

Scipio Cuccagna, Masaya Maeda. A survey on asymptotic stability of ground states of nonlinear Schrödinger equations II. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020450
[17]

Junyong Eom, Kazuhiro Ishige. Large time behavior of ODE type solutions to nonlinear diffusion equations. Discrete & Continuous Dynamical Systems - A, 2020, 40 (6) : 3395-3409. doi: 10.3934/dcds.2019229
[18]

Nguyen Huu Can, Nguyen Huy Tuan, Donal O'Regan, Vo Van Au. On a final value problem for a class of nonlinear hyperbolic equations with damping term. Evolution Equations & Control Theory, 2021, 10 (1) : 103-127. doi: 10.3934/eect.2020053
[19]

Andrew Comech, Scipio Cuccagna. On asymptotic stability of ground states of some systems of nonlinear Schrödinger equations. Discrete & Continuous Dynamical Systems - A, 2021, 41 (3) : 1225-1270. doi: 10.3934/dcds.2020316
[20]

Marion Darbas, Jérémy Heleine, Stephanie Lohrengel. Numerical resolution by the quasi-reversibility method of a data completion problem for Maxwell's equations. Inverse Problems & Imaging, 2020, 14 (6) : 1107-1133. doi: 10.3934/ipi.2020056

 Impact Factor: 

Tools

Metrics

  • PDF downloads (58)
  • HTML views (0)
  • Cited by (1)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences