October  2013, 9(4): 893-899. doi: 10.3934/jimo.2013.9.893

On the strong convergence of a modified Hestenes-Stiefel method for nonconvex optimization

1. 

Department of Mathematics, Changsha University of Science and Technology, Changsha 410004, China, China

Received  November 2012 Revised  February 2013 Published  August 2013

In [8], Zhang et al. proposed a modified three-term HS (MTTHS) conjugate gradient method and proved that this method converges globally for nonconvex minimization in the sense that $\liminf_{k\to\infty}\|\nabla f(x_k)\|=0$ when the Armijo or Wolfe line search is used. In this paper, we further study the convergence property of the MTTHS method. We show that the MTTHS method has strongly global convergence property (i.e., $\lim_{k\to\infty}\|\nabla f(x_k)\|=0$) for nonconvex optimization by the use of the backtracking type line search in [7]. Some preliminary numerical results are reported.
Citation: Weijun Zhou, Youhua Zhou. On the strong convergence of a modified Hestenes-Stiefel method for nonconvex optimization. Journal of Industrial & Management Optimization, 2013, 9 (4) : 893-899. doi: 10.3934/jimo.2013.9.893
References:
[1]

M. R. Hestenes and E. L. Stiefel, Method of conjugate gradient for solving linear systems,, J. Res. Nat. Bur. Stand., 49 (1952), 409.

[2]

D. Li and M. Fukushima, A modified BFGS method and its global convergence in nonconvex minimization,, J. Comput. Appl. Math., 129 (2001), 15. doi: 10.1016/S0377-0427(00)00540-9.

[3]

D. Li and M. Fukushima, On the global convergence of the BFGS method for nonconvex unconstrained optimization problems,, SIAM J. Optim., 11 (2001), 1054. doi: 10.1137/S1052623499354242.

[4]

J. J. Moré, B. S. Garbow and K. H. Hillstrom, Testing unconstrained optimization software,, ACM Trans. Math. Softw., 7 (1981), 17. doi: 10.1145/355934.355936.

[5]

E. Polak and G. Ribière, Note sur la convergence de méthodes de directions conjuguées,, Rev. Fr. Inform. Rech. Oper., 16 (1969), 35.

[6]

B. T. Polyak, The conjugate gradient method in extreme problems,, USSR Comput. Math. Math. Phys., 9 (1969), 94. doi: 10.1016/0041-5553(69)90035-4.

[7]

L. Zhang, W. Zhou and D. Li, A descent modified Polak-Ribière-Polyak conjugate gradient method and its global convergence,, IMA J. Numer. Anal., 26 (2006), 629. doi: 10.1093/imanum/drl016.

[8]

L. Zhang, W. Zhou and D. Li, Some descent three-term conjugate gradient methods and their global convergence,, Optim. Meth. Softw., 22 (2007), 697. doi: 10.1080/10556780701223293.

[9]

W. Zhou and D. Li, On the convergence properties of the unmodified PRP method with a non-descent line search,, submitted., (). doi: 10.1080/10556788.2013.811241.

show all references

References:
[1]

M. R. Hestenes and E. L. Stiefel, Method of conjugate gradient for solving linear systems,, J. Res. Nat. Bur. Stand., 49 (1952), 409.

[2]

D. Li and M. Fukushima, A modified BFGS method and its global convergence in nonconvex minimization,, J. Comput. Appl. Math., 129 (2001), 15. doi: 10.1016/S0377-0427(00)00540-9.

[3]

D. Li and M. Fukushima, On the global convergence of the BFGS method for nonconvex unconstrained optimization problems,, SIAM J. Optim., 11 (2001), 1054. doi: 10.1137/S1052623499354242.

[4]

J. J. Moré, B. S. Garbow and K. H. Hillstrom, Testing unconstrained optimization software,, ACM Trans. Math. Softw., 7 (1981), 17. doi: 10.1145/355934.355936.

[5]

E. Polak and G. Ribière, Note sur la convergence de méthodes de directions conjuguées,, Rev. Fr. Inform. Rech. Oper., 16 (1969), 35.

[6]

B. T. Polyak, The conjugate gradient method in extreme problems,, USSR Comput. Math. Math. Phys., 9 (1969), 94. doi: 10.1016/0041-5553(69)90035-4.

[7]

L. Zhang, W. Zhou and D. Li, A descent modified Polak-Ribière-Polyak conjugate gradient method and its global convergence,, IMA J. Numer. Anal., 26 (2006), 629. doi: 10.1093/imanum/drl016.

[8]

L. Zhang, W. Zhou and D. Li, Some descent three-term conjugate gradient methods and their global convergence,, Optim. Meth. Softw., 22 (2007), 697. doi: 10.1080/10556780701223293.

[9]

W. Zhou and D. Li, On the convergence properties of the unmodified PRP method with a non-descent line search,, submitted., (). doi: 10.1080/10556788.2013.811241.

[1]

Yves Bourgault, Damien Broizat, Pierre-Emmanuel Jabin. Convergence rate for the method of moments with linear closure relations. Kinetic & Related Models, 2015, 8 (1) : 1-27. doi: 10.3934/krm.2015.8.1

[2]

Yu-Ning Yang, Su Zhang. On linear convergence of projected gradient method for a class of affine rank minimization problems. Journal of Industrial & Management Optimization, 2016, 12 (4) : 1507-1519. doi: 10.3934/jimo.2016.12.1507

[3]

Wei-Zhe Gu, Li-Yong Lu. The linear convergence of a derivative-free descent method for nonlinear complementarity problems. Journal of Industrial & Management Optimization, 2017, 13 (2) : 531-548. doi: 10.3934/jimo.2016030

[4]

Zhili Ge, Gang Qian, Deren Han. Global convergence of an inexact operator splitting method for monotone variational inequalities. Journal of Industrial & Management Optimization, 2011, 7 (4) : 1013-1026. doi: 10.3934/jimo.2011.7.1013

[5]

Liyan Qi, Xiantao Xiao, Liwei Zhang. On the global convergence of a parameter-adjusting Levenberg-Marquardt method. Numerical Algebra, Control & Optimization, 2015, 5 (1) : 25-36. doi: 10.3934/naco.2015.5.25

[6]

Leong-Kwan Li, Sally Shao. Convergence analysis of the weighted state space search algorithm for recurrent neural networks. Numerical Algebra, Control & Optimization, 2014, 4 (3) : 193-207. doi: 10.3934/naco.2014.4.193

[7]

Zheng-Hai Huang, Nan Lu. Global and global linear convergence of smoothing algorithm for the Cartesian $P_*(\kappa)$-SCLCP. Journal of Industrial & Management Optimization, 2012, 8 (1) : 67-86. doi: 10.3934/jimo.2012.8.67

[8]

Liu Liu. Uniform spectral convergence of the stochastic Galerkin method for the linear semiconductor Boltzmann equation with random inputs and diffusive scaling. Kinetic & Related Models, 2018, 11 (5) : 1139-1156. doi: 10.3934/krm.2018044

[9]

Nora Merabet. Global convergence of a memory gradient method with closed-form step size formula. Conference Publications, 2007, 2007 (Special) : 721-730. doi: 10.3934/proc.2007.2007.721

[10]

Matthias Gerdts, Stefan Horn, Sven-Joachim Kimmerle. Line search globalization of a semismooth Newton method for operator equations in Hilbert spaces with applications in optimal control. Journal of Industrial & Management Optimization, 2017, 13 (1) : 47-62. doi: 10.3934/jimo.2016003

[11]

Wilhelm Schlag. Regularity and convergence rates for the Lyapunov exponents of linear cocycles. Journal of Modern Dynamics, 2013, 7 (4) : 619-637. doi: 10.3934/jmd.2013.7.619

[12]

Feng-Yu Wang. Exponential convergence of non-linear monotone SPDEs. Discrete & Continuous Dynamical Systems - A, 2015, 35 (11) : 5239-5253. doi: 10.3934/dcds.2015.35.5239

[13]

G. Gentile, V. Mastropietro. Convergence of Lindstedt series for the non linear wave equation. Communications on Pure & Applied Analysis, 2004, 3 (3) : 509-514. doi: 10.3934/cpaa.2004.3.509

[14]

Jinyan Fan, Jianyu Pan. On the convergence rate of the inexact Levenberg-Marquardt method. Journal of Industrial & Management Optimization, 2011, 7 (1) : 199-210. doi: 10.3934/jimo.2011.7.199

[15]

Stefan Kindermann. Convergence of the gradient method for ill-posed problems. Inverse Problems & Imaging, 2017, 11 (4) : 703-720. doi: 10.3934/ipi.2017033

[16]

Regina S. Burachik, C. Yalçın Kaya. An update rule and a convergence result for a penalty function method. Journal of Industrial & Management Optimization, 2007, 3 (2) : 381-398. doi: 10.3934/jimo.2007.3.381

[17]

Karl Kunisch, Markus Müller. Uniform convergence of the POD method and applications to optimal control. Discrete & Continuous Dynamical Systems - A, 2015, 35 (9) : 4477-4501. doi: 10.3934/dcds.2015.35.4477

[18]

Yong Duan, Jian-Guo Liu. Convergence analysis of the vortex blob method for the $b$-equation. Discrete & Continuous Dynamical Systems - A, 2014, 34 (5) : 1995-2011. doi: 10.3934/dcds.2014.34.1995

[19]

Chun-Hsiung Hsia, Chang-Yeol Jung, Bongsuk Kwon. On the global convergence of frequency synchronization for Kuramoto and Winfree oscillators. Discrete & Continuous Dynamical Systems - B, 2019, 24 (7) : 3319-3334. doi: 10.3934/dcdsb.2018322

[20]

Yanxing Cui, Chuanlong Wang, Ruiping Wen. On the convergence of generalized parallel multisplitting iterative methods for semidefinite linear systems. Numerical Algebra, Control & Optimization, 2012, 2 (4) : 863-873. doi: 10.3934/naco.2012.2.863

2017 Impact Factor: 0.994

Metrics

  • PDF downloads (7)
  • HTML views (0)
  • Cited by (2)

Other articles
by authors

[Back to Top]