American Institute of Mathematical Sciences

Advanced
July  2021, 17(4): 2265-2275. doi: 10.3934/jimo.2020068

Convergence properties of inexact Levenberg-Marquardt method under Hölderian local error bound

Haiyan Wang 1, and  Jinyan Fan 2,,

1. 

School of Mathematical Sciences, Shanghai Jiao Tong University, Shanghai 200240, China
2. 

School of Mathematical Sciences, Shanghai Jiao Tong University, Key Lab of Scientific and Engineering Computing (Ministry of Education), Shanghai 200240, China

* Corresponding author: Jinyan Fan

Received  August 2019 Revised  November 2019 Published  July 2021 Early access  March 2020

Fund Project: The authors are supported by Chinese NSF grants 11971309

In this paper, we study convergence properties of the inexact Levenberg-Marquardt method under the Hölderian local error bound condition and the Hölderian continuity of the Jacobian. The formula of the convergence rates are given, which are functions with respect to the Levenberg-Marquardt parameter, the perturbation vector, as well as the orders of the Hölderian local error bound and Hölderian continuity of the Jacobian.

Keywords: Nonlinear equations, inexact Levenberg-Marquardt method, Hölderian local error bound, Hölderian continuity, convergence rate.
Mathematics Subject Classification: 65K05, 90C30.
Citation: Haiyan Wang, Jinyan Fan. Convergence properties of inexact Levenberg-Marquardt method under Hölderian local error bound. Journal of Industrial & Management Optimization, 2021, 17 (4) : 2265-2275. doi: 10.3934/jimo.2020068
References:
[1]

M. Ahookhosh, F. J. Aragón, R. M. T. Fleming and P. T. Vuong, Local convergence of Levenberg-Marquardt methods under Hölderian metric subregularity, Adv. Comput. Math., 45 (2019), 2771–2806, arXiv: 1703.07461. doi: 10.1007/s10444-019-09708-7.  Google Scholar

[2]

H. Dan, N. Yamashita and M. Fukushima, Convergence properties of the inexact Levenberg-Marquardt method under local error bound, Optimization Methods and Software, 17 (2002), 605-626.  doi: 10.1080/1055678021000049345.  Google Scholar

[3]

F. Facchinei and C. Kanzow, A nonsmooth inexact Newton method for the solution of large-scale nonlinear complementarity problems, Mathematical Programming, 76 (1997), 493-512.  doi: 10.1007/BF02614395.  Google Scholar

[4]

J. Y. Fan, A modified Levenberg-Marquardt algorithm for singular system of nonlinear equations, Journal of Computational Mathematics, 21 (2003), 625-636.   Google Scholar

[5]

J. Y. Fan, The modified Levenberg-Marquardt method for nonlinear equations with cubic convergence, Mathematics of Computation, 81 (2012), 447-466.  doi: 10.1090/S0025-5718-2011-02496-8.  Google Scholar

[6]

J. Y. Fan, J. C. Huang and J. Y. Pan, An adaptive multi-step Levenberg-Marquardt method, Journal of Scientific Computing, 78 (2019), 531-548.  doi: 10.1007/s10915-018-0777-8.  Google Scholar

[7]

J. Y. Fan and J. Y. Pan, Inexact Levenberg-Marquardt method for nonlinear equations, Discrete Continuous Dynamical System-Series B, 4 (2004), 1223-1232.  doi: 10.3934/dcdsb.2004.4.1223.  Google Scholar

[8]

J. Y. Fan and J. Y. Pan, A note on the Levenberg-Marquardt parameter, Applied Mathematics and Computation, 207 (2009), 351-359.  doi: 10.1016/j.amc.2008.10.056.  Google Scholar

[9]

J. Y. Fan and J. Y. Pan, On the convergence rate of the inexact Levenberg-Marquardt method, Industrial and Management Optimization, 7 (2011), 199-210.  doi: 10.3934/jimo.2011.7.199.  Google Scholar

[10]

J. Y. Fan and Y. X. Yuan, On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption, Computing, 74 (2005), 23-39.  doi: 10.1007/s00607-004-0083-1.  Google Scholar

[11]

A. Fischera, P. K. Shuklaa and M. Wang, On the inexactness level of robust Levenberg-Marquardt methods, Optimization, 59 (2010), 273-287.  doi: 10.1080/02331930801951256.  Google Scholar

[12]

C. T. Kelley, Solving Nonlinear Equations with Newton's Method, Fundamentals of Algorithms, SIAM, Philadelphia, 2003. doi: 10.1137/1.9780898718898.  Google Scholar

[13]

K. Levenberg, A method for the solution of certain nonlinear problems in least squares, Quart. Appl. Math., 2 (1944), 164-168.  doi: 10.1090/qam/10666.  Google Scholar

[14]

D. W. Marquardt, An algorithm for least-squares estimation of nonlinear inequalities, SIAM J. Appl. Math., 11 (1963), 431-441.  doi: 10.1137/0111030.  Google Scholar

[15]

J. J. Moré, The Levenberg-Marquardt algorithm: implementation and theory, In: G. A. Watson, ed., Lecture Notes in Mathematics 630: Numerical Analysis, Springer-Verlag, Berlin, 1978, 105–116.  Google Scholar

[16]

M. J. D. Powell, Convergence properties of a class of minimization algorithms, Nonlinear Programming, 2 (1974), 1-27.   Google Scholar

[17]

G. W. Stewart and J.-G. Sun, Matrix Perturbation Theory, (Computer Science and Scientific Computing), Academic Press Boston, 1990.  Google Scholar

[18]

H. Y. Wang and J. Y. Fan, Convergence rate of the Levenberg-Marquardt method under hölderian local error bound, Optimization Methods and Software, 2019. doi: 10.1080/10556788.2019.1694927.  Google Scholar

[19]

N. Yamashita and M. Fukushima, On the rate of convergence of the Levenberg-Marquardt method, Computing, (15) (2001), 239-249.  doi: 10.1007/978-3-7091-6217-0_18.  Google Scholar

[20]

Y. X. Yuan, Recent advances in trust region algorithms, Math. Program., Ser. B, 151 (2015), 249–281. doi: 10.1007/s10107-015-0893-2.  Google Scholar

[21]

X. D. Zhu and G. H. Lin, Improved convergence results for a modified Levenberg-Marquardt method for nonlinear equations and applications in MPCC, Optimization Methods and Software, 31 (2016), 791-804.  doi: 10.1080/10556788.2016.1171863.  Google Scholar

show all references

References:
[1]

M. Ahookhosh, F. J. Aragón, R. M. T. Fleming and P. T. Vuong, Local convergence of Levenberg-Marquardt methods under Hölderian metric subregularity, Adv. Comput. Math., 45 (2019), 2771–2806, arXiv: 1703.07461. doi: 10.1007/s10444-019-09708-7.  Google Scholar

[2]

H. Dan, N. Yamashita and M. Fukushima, Convergence properties of the inexact Levenberg-Marquardt method under local error bound, Optimization Methods and Software, 17 (2002), 605-626.  doi: 10.1080/1055678021000049345.  Google Scholar

[3]

F. Facchinei and C. Kanzow, A nonsmooth inexact Newton method for the solution of large-scale nonlinear complementarity problems, Mathematical Programming, 76 (1997), 493-512.  doi: 10.1007/BF02614395.  Google Scholar

[4]

J. Y. Fan, A modified Levenberg-Marquardt algorithm for singular system of nonlinear equations, Journal of Computational Mathematics, 21 (2003), 625-636.   Google Scholar

[5]

J. Y. Fan, The modified Levenberg-Marquardt method for nonlinear equations with cubic convergence, Mathematics of Computation, 81 (2012), 447-466.  doi: 10.1090/S0025-5718-2011-02496-8.  Google Scholar

[6]

J. Y. Fan, J. C. Huang and J. Y. Pan, An adaptive multi-step Levenberg-Marquardt method, Journal of Scientific Computing, 78 (2019), 531-548.  doi: 10.1007/s10915-018-0777-8.  Google Scholar

[7]

J. Y. Fan and J. Y. Pan, Inexact Levenberg-Marquardt method for nonlinear equations, Discrete Continuous Dynamical System-Series B, 4 (2004), 1223-1232.  doi: 10.3934/dcdsb.2004.4.1223.  Google Scholar

[8]

J. Y. Fan and J. Y. Pan, A note on the Levenberg-Marquardt parameter, Applied Mathematics and Computation, 207 (2009), 351-359.  doi: 10.1016/j.amc.2008.10.056.  Google Scholar

[9]

J. Y. Fan and J. Y. Pan, On the convergence rate of the inexact Levenberg-Marquardt method, Industrial and Management Optimization, 7 (2011), 199-210.  doi: 10.3934/jimo.2011.7.199.  Google Scholar

[10]

J. Y. Fan and Y. X. Yuan, On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption, Computing, 74 (2005), 23-39.  doi: 10.1007/s00607-004-0083-1.  Google Scholar

[11]

A. Fischera, P. K. Shuklaa and M. Wang, On the inexactness level of robust Levenberg-Marquardt methods, Optimization, 59 (2010), 273-287.  doi: 10.1080/02331930801951256.  Google Scholar

[12]

C. T. Kelley, Solving Nonlinear Equations with Newton's Method, Fundamentals of Algorithms, SIAM, Philadelphia, 2003. doi: 10.1137/1.9780898718898.  Google Scholar

[13]

K. Levenberg, A method for the solution of certain nonlinear problems in least squares, Quart. Appl. Math., 2 (1944), 164-168.  doi: 10.1090/qam/10666.  Google Scholar

[14]

D. W. Marquardt, An algorithm for least-squares estimation of nonlinear inequalities, SIAM J. Appl. Math., 11 (1963), 431-441.  doi: 10.1137/0111030.  Google Scholar

[15]

J. J. Moré, The Levenberg-Marquardt algorithm: implementation and theory, In: G. A. Watson, ed., Lecture Notes in Mathematics 630: Numerical Analysis, Springer-Verlag, Berlin, 1978, 105–116.  Google Scholar

[16]

M. J. D. Powell, Convergence properties of a class of minimization algorithms, Nonlinear Programming, 2 (1974), 1-27.   Google Scholar

[17]

G. W. Stewart and J.-G. Sun, Matrix Perturbation Theory, (Computer Science and Scientific Computing), Academic Press Boston, 1990.  Google Scholar

[18]

H. Y. Wang and J. Y. Fan, Convergence rate of the Levenberg-Marquardt method under hölderian local error bound, Optimization Methods and Software, 2019. doi: 10.1080/10556788.2019.1694927.  Google Scholar

[19]

N. Yamashita and M. Fukushima, On the rate of convergence of the Levenberg-Marquardt method, Computing, (15) (2001), 239-249.  doi: 10.1007/978-3-7091-6217-0_18.  Google Scholar

[20]

Y. X. Yuan, Recent advances in trust region algorithms, Math. Program., Ser. B, 151 (2015), 249–281. doi: 10.1007/s10107-015-0893-2.  Google Scholar

[21]

X. D. Zhu and G. H. Lin, Improved convergence results for a modified Levenberg-Marquardt method for nonlinear equations and applications in MPCC, Optimization Methods and Software, 31 (2016), 791-804.  doi: 10.1080/10556788.2016.1171863.  Google Scholar

[1]

Jinyan Fan, Jianyu Pan. On the convergence rate of the inexact Levenberg-Marquardt method. Journal of Industrial & Management Optimization, 2011, 7 (1) : 199-210. doi: 10.3934/jimo.2011.7.199
[2]

Jinyan Fan, Jianyu Pan. Inexact Levenberg-Marquardt method for nonlinear equations. Discrete & Continuous Dynamical Systems - B, 2004, 4 (4) : 1223-1232. doi: 10.3934/dcdsb.2004.4.1223
[3]

Liyan Qi, Xiantao Xiao, Liwei Zhang. On the global convergence of a parameter-adjusting Levenberg-Marquardt method. Numerical Algebra, Control & Optimization, 2015, 5 (1) : 25-36. doi: 10.3934/naco.2015.5.25
[4]

Jinyan Fan. On the Levenberg-Marquardt methods for convex constrained nonlinear equations. Journal of Industrial & Management Optimization, 2013, 9 (1) : 227-241. doi: 10.3934/jimo.2013.9.227
[5]

Kyudong Choi. Persistence of Hölder continuity for non-local integro-differential equations. Discrete & Continuous Dynamical Systems, 2013, 33 (5) : 1741-1771. doi: 10.3934/dcds.2013.33.1741
[6]

Johann Baumeister, Barbara Kaltenbacher, Antonio Leitão. On Levenberg-Marquardt-Kaczmarz iterative methods for solving systems of nonlinear ill-posed equations. Inverse Problems & Imaging, 2010, 4 (3) : 335-350. doi: 10.3934/ipi.2010.4.335
[7]

Wei Ouyang, Li Li. Hölder strong metric subregularity and its applications to convergence analysis of inexact Newton methods. Journal of Industrial & Management Optimization, 2021, 17 (1) : 169-184. doi: 10.3934/jimo.2019105
[8]

Lili Li, Chunrong Chen. Nonlinear scalarization with applications to Hölder continuity of approximate solutions. Numerical Algebra, Control & Optimization, 2014, 4 (4) : 295-307. doi: 10.3934/naco.2014.4.295
[9]

Arnulf Jentzen, Benno Kuckuck, Thomas Müller-Gronbach, Larisa Yaroslavtseva. Counterexamples to local Lipschitz and local Hölder continuity with respect to the initial values for additive noise driven stochastic differential equations with smooth drift coefficient functions with at most polynomially growing derivatives. Discrete & Continuous Dynamical Systems - B, 2021  doi: 10.3934/dcdsb.2021203
[10]

Woocheol Choi, Youngwoo Koh. On the splitting method for the nonlinear Schrödinger equation with initial data in $ H^1 $. Discrete & Continuous Dynamical Systems, 2021, 41 (8) : 3837-3867. doi: 10.3934/dcds.2021019
[11]

Luis Silvestre. Hölder continuity for integro-differential parabolic equations with polynomial growth respect to the gradient. Discrete & Continuous Dynamical Systems, 2010, 28 (3) : 1069-1081. doi: 10.3934/dcds.2010.28.1069
[12]

John B. Greer, Andrea L. Bertozzi. $H^1$ Solutions of a class of fourth order nonlinear equations for image processing. Discrete & Continuous Dynamical Systems, 2004, 10 (1&2) : 349-366. doi: 10.3934/dcds.2004.10.349
[13]

Junichi Harada, Mitsuharu Ôtani. $H^2$-solutions for some elliptic equations with nonlinear boundary conditions. Conference Publications, 2009, 2009 (Special) : 333-339. doi: 10.3934/proc.2009.2009.333
[14]

Samia Challal, Abdeslem Lyaghfouri. Hölder continuity of solutions to the $A$-Laplace equation involving measures. Communications on Pure & Applied Analysis, 2009, 8 (5) : 1577-1583. doi: 10.3934/cpaa.2009.8.1577
[15]

Zhili Ge, Gang Qian, Deren Han. Global convergence of an inexact operator splitting method for monotone variational inequalities. Journal of Industrial & Management Optimization, 2011, 7 (4) : 1013-1026. doi: 10.3934/jimo.2011.7.1013
[16]

Lin Du, Yun Zhang. $\mathcal{H}_∞$ filtering for switched nonlinear systems: A state projection method. Journal of Industrial & Management Optimization, 2018, 14 (1) : 19-33. doi: 10.3934/jimo.2017035
[17]

Wen-ling Zhao, Dao-jin Song. A global error bound via the SQP method for constrained optimization problem. Journal of Industrial & Management Optimization, 2007, 3 (4) : 775-781. doi: 10.3934/jimo.2007.3.775
[18]

Hongyong Cui, Peter E. Kloeden, Wenqiang Zhao. Strong $ (L^2,L^\gamma\cap H_0^1) $-continuity in initial data of nonlinear reaction-diffusion equation in any space dimension. Electronic Research Archive, 2020, 28 (3) : 1357-1374. doi: 10.3934/era.2020072
[19]

Jie-Wen He, Chi-Chon Lei, Chen-Yang Shi, Seak-Weng Vong. An inexact alternating direction method of multipliers for a kind of nonlinear complementarity problems. Numerical Algebra, Control & Optimization, 2021, 11 (3) : 353-362. doi: 10.3934/naco.2020030
[20]

Sanghoon Kwon, Seonhee Lim. Equidistribution with an error rate and Diophantine approximation over a local field of positive characteristic. Discrete & Continuous Dynamical Systems, 2018, 38 (1) : 169-186. doi: 10.3934/dcds.2018008

2020 Impact Factor: 1.801

Tools

Metrics

  • PDF downloads (199)
  • HTML views (599)
  • Cited by (0)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences