American Institute of Mathematical Sciences

Advanced
July  2021, 17(4): 2265-2275. doi: 10.3934/jimo.2020068

Convergence properties of inexact Levenberg-Marquardt method under Hölderian local error bound

Haiyan Wang 1, and  Jinyan Fan 2,,

1. 

School of Mathematical Sciences, Shanghai Jiao Tong University, Shanghai 200240, China
2. 

School of Mathematical Sciences, Shanghai Jiao Tong University, Key Lab of Scientific and Engineering Computing (Ministry of Education), Shanghai 200240, China

* Corresponding author: Jinyan Fan

Received  August 2019 Revised  November 2019 Published  July 2021 Early access  March 2020

Fund Project: The authors are supported by Chinese NSF grants 11971309

In this paper, we study convergence properties of the inexact Levenberg-Marquardt method under the Hölderian local error bound condition and the Hölderian continuity of the Jacobian. The formula of the convergence rates are given, which are functions with respect to the Levenberg-Marquardt parameter, the perturbation vector, as well as the orders of the Hölderian local error bound and Hölderian continuity of the Jacobian.

Keywords: Nonlinear equations, inexact Levenberg-Marquardt method, Hölderian local error bound, Hölderian continuity, convergence rate.
Mathematics Subject Classification: 65K05, 90C30.
Citation: Haiyan Wang, Jinyan Fan. Convergence properties of inexact Levenberg-Marquardt method under Hölderian local error bound. Journal of Industrial and Management Optimization, 2021, 17 (4) : 2265-2275. doi: 10.3934/jimo.2020068
References:
[1]

M. Ahookhosh, F. J. Aragón, R. M. T. Fleming and P. T. Vuong, Local convergence of Levenberg-Marquardt methods under Hölderian metric subregularity, Adv. Comput. Math., 45 (2019), 2771–2806, arXiv: 1703.07461. doi: 10.1007/s10444-019-09708-7.

[2]

H. Dan, N. Yamashita and M. Fukushima, Convergence properties of the inexact Levenberg-Marquardt method under local error bound, Optimization Methods and Software, 17 (2002), 605-626.  doi: 10.1080/1055678021000049345.

[3]

F. Facchinei and C. Kanzow, A nonsmooth inexact Newton method for the solution of large-scale nonlinear complementarity problems, Mathematical Programming, 76 (1997), 493-512.  doi: 10.1007/BF02614395.

[4]

J. Y. Fan, A modified Levenberg-Marquardt algorithm for singular system of nonlinear equations, Journal of Computational Mathematics, 21 (2003), 625-636. 

[5]

J. Y. Fan, The modified Levenberg-Marquardt method for nonlinear equations with cubic convergence, Mathematics of Computation, 81 (2012), 447-466.  doi: 10.1090/S0025-5718-2011-02496-8.

[6]

J. Y. Fan, J. C. Huang and J. Y. Pan, An adaptive multi-step Levenberg-Marquardt method, Journal of Scientific Computing, 78 (2019), 531-548.  doi: 10.1007/s10915-018-0777-8.

[7]

J. Y. Fan and J. Y. Pan, Inexact Levenberg-Marquardt method for nonlinear equations, Discrete Continuous Dynamical System-Series B, 4 (2004), 1223-1232.  doi: 10.3934/dcdsb.2004.4.1223.

[8]

J. Y. Fan and J. Y. Pan, A note on the Levenberg-Marquardt parameter, Applied Mathematics and Computation, 207 (2009), 351-359.  doi: 10.1016/j.amc.2008.10.056.

[9]

J. Y. Fan and J. Y. Pan, On the convergence rate of the inexact Levenberg-Marquardt method, Industrial and Management Optimization, 7 (2011), 199-210.  doi: 10.3934/jimo.2011.7.199.

[10]

J. Y. Fan and Y. X. Yuan, On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption, Computing, 74 (2005), 23-39.  doi: 10.1007/s00607-004-0083-1.

[11]

A. Fischera, P. K. Shuklaa and M. Wang, On the inexactness level of robust Levenberg-Marquardt methods, Optimization, 59 (2010), 273-287.  doi: 10.1080/02331930801951256.

[12]

C. T. Kelley, Solving Nonlinear Equations with Newton's Method, Fundamentals of Algorithms, SIAM, Philadelphia, 2003. doi: 10.1137/1.9780898718898.

[13]

K. Levenberg, A method for the solution of certain nonlinear problems in least squares, Quart. Appl. Math., 2 (1944), 164-168.  doi: 10.1090/qam/10666.

[14]

D. W. Marquardt, An algorithm for least-squares estimation of nonlinear inequalities, SIAM J. Appl. Math., 11 (1963), 431-441.  doi: 10.1137/0111030.

[15]

J. J. Moré, The Levenberg-Marquardt algorithm: implementation and theory, In: G. A. Watson, ed., Lecture Notes in Mathematics 630: Numerical Analysis, Springer-Verlag, Berlin, 1978, 105–116.

[16]

M. J. D. Powell, Convergence properties of a class of minimization algorithms, Nonlinear Programming, 2 (1974), 1-27. 

[17]

G. W. Stewart and J.-G. Sun, Matrix Perturbation Theory, (Computer Science and Scientific Computing), Academic Press Boston, 1990.

[18]

H. Y. Wang and J. Y. Fan, Convergence rate of the Levenberg-Marquardt method under hölderian local error bound, Optimization Methods and Software, 2019. doi: 10.1080/10556788.2019.1694927.

[19]

N. Yamashita and M. Fukushima, On the rate of convergence of the Levenberg-Marquardt method, Computing, (15) (2001), 239-249.  doi: 10.1007/978-3-7091-6217-0_18.

[20]

Y. X. Yuan, Recent advances in trust region algorithms, Math. Program., Ser. B, 151 (2015), 249–281. doi: 10.1007/s10107-015-0893-2.

[21]

X. D. Zhu and G. H. Lin, Improved convergence results for a modified Levenberg-Marquardt method for nonlinear equations and applications in MPCC, Optimization Methods and Software, 31 (2016), 791-804.  doi: 10.1080/10556788.2016.1171863.

show all references

References:
[1]

M. Ahookhosh, F. J. Aragón, R. M. T. Fleming and P. T. Vuong, Local convergence of Levenberg-Marquardt methods under Hölderian metric subregularity, Adv. Comput. Math., 45 (2019), 2771–2806, arXiv: 1703.07461. doi: 10.1007/s10444-019-09708-7.

[2]

H. Dan, N. Yamashita and M. Fukushima, Convergence properties of the inexact Levenberg-Marquardt method under local error bound, Optimization Methods and Software, 17 (2002), 605-626.  doi: 10.1080/1055678021000049345.

[3]

F. Facchinei and C. Kanzow, A nonsmooth inexact Newton method for the solution of large-scale nonlinear complementarity problems, Mathematical Programming, 76 (1997), 493-512.  doi: 10.1007/BF02614395.

[4]

J. Y. Fan, A modified Levenberg-Marquardt algorithm for singular system of nonlinear equations, Journal of Computational Mathematics, 21 (2003), 625-636. 

[5]

J. Y. Fan, The modified Levenberg-Marquardt method for nonlinear equations with cubic convergence, Mathematics of Computation, 81 (2012), 447-466.  doi: 10.1090/S0025-5718-2011-02496-8.

[6]

J. Y. Fan, J. C. Huang and J. Y. Pan, An adaptive multi-step Levenberg-Marquardt method, Journal of Scientific Computing, 78 (2019), 531-548.  doi: 10.1007/s10915-018-0777-8.

[7]

J. Y. Fan and J. Y. Pan, Inexact Levenberg-Marquardt method for nonlinear equations, Discrete Continuous Dynamical System-Series B, 4 (2004), 1223-1232.  doi: 10.3934/dcdsb.2004.4.1223.

[8]

J. Y. Fan and J. Y. Pan, A note on the Levenberg-Marquardt parameter, Applied Mathematics and Computation, 207 (2009), 351-359.  doi: 10.1016/j.amc.2008.10.056.

[9]

J. Y. Fan and J. Y. Pan, On the convergence rate of the inexact Levenberg-Marquardt method, Industrial and Management Optimization, 7 (2011), 199-210.  doi: 10.3934/jimo.2011.7.199.

[10]

J. Y. Fan and Y. X. Yuan, On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption, Computing, 74 (2005), 23-39.  doi: 10.1007/s00607-004-0083-1.

[11]

A. Fischera, P. K. Shuklaa and M. Wang, On the inexactness level of robust Levenberg-Marquardt methods, Optimization, 59 (2010), 273-287.  doi: 10.1080/02331930801951256.

[12]

C. T. Kelley, Solving Nonlinear Equations with Newton's Method, Fundamentals of Algorithms, SIAM, Philadelphia, 2003. doi: 10.1137/1.9780898718898.

[13]

K. Levenberg, A method for the solution of certain nonlinear problems in least squares, Quart. Appl. Math., 2 (1944), 164-168.  doi: 10.1090/qam/10666.

[14]

D. W. Marquardt, An algorithm for least-squares estimation of nonlinear inequalities, SIAM J. Appl. Math., 11 (1963), 431-441.  doi: 10.1137/0111030.

[15]

J. J. Moré, The Levenberg-Marquardt algorithm: implementation and theory, In: G. A. Watson, ed., Lecture Notes in Mathematics 630: Numerical Analysis, Springer-Verlag, Berlin, 1978, 105–116.

[16]

M. J. D. Powell, Convergence properties of a class of minimization algorithms, Nonlinear Programming, 2 (1974), 1-27. 

[17]

G. W. Stewart and J.-G. Sun, Matrix Perturbation Theory, (Computer Science and Scientific Computing), Academic Press Boston, 1990.

[18]

H. Y. Wang and J. Y. Fan, Convergence rate of the Levenberg-Marquardt method under hölderian local error bound, Optimization Methods and Software, 2019. doi: 10.1080/10556788.2019.1694927.

[19]

N. Yamashita and M. Fukushima, On the rate of convergence of the Levenberg-Marquardt method, Computing, (15) (2001), 239-249.  doi: 10.1007/978-3-7091-6217-0_18.

[20]

Y. X. Yuan, Recent advances in trust region algorithms, Math. Program., Ser. B, 151 (2015), 249–281. doi: 10.1007/s10107-015-0893-2.

[21]

X. D. Zhu and G. H. Lin, Improved convergence results for a modified Levenberg-Marquardt method for nonlinear equations and applications in MPCC, Optimization Methods and Software, 31 (2016), 791-804.  doi: 10.1080/10556788.2016.1171863.

[1]

Jinyan Fan, Jianyu Pan. On the convergence rate of the inexact Levenberg-Marquardt method. Journal of Industrial and Management Optimization, 2011, 7 (1) : 199-210. doi: 10.3934/jimo.2011.7.199
[2]

Jinyan Fan, Jianyu Pan. Inexact Levenberg-Marquardt method for nonlinear equations. Discrete and Continuous Dynamical Systems - B, 2004, 4 (4) : 1223-1232. doi: 10.3934/dcdsb.2004.4.1223
[3]

Jirui Ma, Jinyan Fan. On convergence properties of the modified trust region method under Hölderian error bound condition. Journal of Industrial and Management Optimization, 2021  doi: 10.3934/jimo.2021222
[4]

Liyan Qi, Xiantao Xiao, Liwei Zhang. On the global convergence of a parameter-adjusting Levenberg-Marquardt method. Numerical Algebra, Control and Optimization, 2015, 5 (1) : 25-36. doi: 10.3934/naco.2015.5.25
[5]

Jinyan Fan. On the Levenberg-Marquardt methods for convex constrained nonlinear equations. Journal of Industrial and Management Optimization, 2013, 9 (1) : 227-241. doi: 10.3934/jimo.2013.9.227
[6]

Xin-He Miao, Kai Yao, Ching-Yu Yang, Jein-Shan Chen. Levenberg-Marquardt method for absolute value equation associated with second-order cone. Numerical Algebra, Control and Optimization, 2022, 12 (1) : 47-61. doi: 10.3934/naco.2021050
[7]

Kyudong Choi. Persistence of Hölder continuity for non-local integro-differential equations. Discrete and Continuous Dynamical Systems, 2013, 33 (5) : 1741-1771. doi: 10.3934/dcds.2013.33.1741
[8]

Johann Baumeister, Barbara Kaltenbacher, Antonio Leitão. On Levenberg-Marquardt-Kaczmarz iterative methods for solving systems of nonlinear ill-posed equations. Inverse Problems and Imaging, 2010, 4 (3) : 335-350. doi: 10.3934/ipi.2010.4.335
[9]

Wei Ouyang, Li Li. Hölder strong metric subregularity and its applications to convergence analysis of inexact Newton methods. Journal of Industrial and Management Optimization, 2021, 17 (1) : 169-184. doi: 10.3934/jimo.2019105
[10]

Lili Li, Chunrong Chen. Nonlinear scalarization with applications to Hölder continuity of approximate solutions. Numerical Algebra, Control and Optimization, 2014, 4 (4) : 295-307. doi: 10.3934/naco.2014.4.295
[11]

Arnulf Jentzen, Benno Kuckuck, Thomas Müller-Gronbach, Larisa Yaroslavtseva. Counterexamples to local Lipschitz and local Hölder continuity with respect to the initial values for additive noise driven stochastic differential equations with smooth drift coefficient functions with at most polynomially growing derivatives. Discrete and Continuous Dynamical Systems - B, 2022, 27 (7) : 3707-3724. doi: 10.3934/dcdsb.2021203
[12]

Woocheol Choi, Youngwoo Koh. On the splitting method for the nonlinear Schrödinger equation with initial data in $ H^1 $. Discrete and Continuous Dynamical Systems, 2021, 41 (8) : 3837-3867. doi: 10.3934/dcds.2021019
[13]

Luis Silvestre. Hölder continuity for integro-differential parabolic equations with polynomial growth respect to the gradient. Discrete and Continuous Dynamical Systems, 2010, 28 (3) : 1069-1081. doi: 10.3934/dcds.2010.28.1069
[14]

John B. Greer, Andrea L. Bertozzi. $H^1$ Solutions of a class of fourth order nonlinear equations for image processing. Discrete and Continuous Dynamical Systems, 2004, 10 (1&2) : 349-366. doi: 10.3934/dcds.2004.10.349
[15]

Junichi Harada, Mitsuharu Ôtani. $H^2$-solutions for some elliptic equations with nonlinear boundary conditions. Conference Publications, 2009, 2009 (Special) : 333-339. doi: 10.3934/proc.2009.2009.333
[16]

Samia Challal, Abdeslem Lyaghfouri. Hölder continuity of solutions to the $A$-Laplace equation involving measures. Communications on Pure and Applied Analysis, 2009, 8 (5) : 1577-1583. doi: 10.3934/cpaa.2009.8.1577
[17]

Zhili Ge, Gang Qian, Deren Han. Global convergence of an inexact operator splitting method for monotone variational inequalities. Journal of Industrial and Management Optimization, 2011, 7 (4) : 1013-1026. doi: 10.3934/jimo.2011.7.1013
[18]

Lin Du, Yun Zhang. $\mathcal{H}_∞$ filtering for switched nonlinear systems: A state projection method. Journal of Industrial and Management Optimization, 2018, 14 (1) : 19-33. doi: 10.3934/jimo.2017035
[19]

Wen-ling Zhao, Dao-jin Song. A global error bound via the SQP method for constrained optimization problem. Journal of Industrial and Management Optimization, 2007, 3 (4) : 775-781. doi: 10.3934/jimo.2007.3.775
[20]

Hongyong Cui, Peter E. Kloeden, Wenqiang Zhao. Strong $ (L^2,L^\gamma\cap H_0^1) $-continuity in initial data of nonlinear reaction-diffusion equation in any space dimension. Electronic Research Archive, 2020, 28 (3) : 1357-1374. doi: 10.3934/era.2020072

2020 Impact Factor: 1.801

Tools

Metrics

  • PDF downloads (300)
  • HTML views (722)
  • Cited by (0)

Other articles
by authors

[Back to Top]

Copyright © 2022 American Institute of Mathematical Sciences | Privacy Policy