October  2013, 9(4): 893-899. doi: 10.3934/jimo.2013.9.893

On the strong convergence of a modified Hestenes-Stiefel method for nonconvex optimization

1. 

Department of Mathematics, Changsha University of Science and Technology, Changsha 410004, China, China

Received  November 2012 Revised  February 2013 Published  August 2013

In [8], Zhang et al. proposed a modified three-term HS (MTTHS) conjugate gradient method and proved that this method converges globally for nonconvex minimization in the sense that $\liminf_{k\to\infty}\|\nabla f(x_k)\|=0$ when the Armijo or Wolfe line search is used. In this paper, we further study the convergence property of the MTTHS method. We show that the MTTHS method has strongly global convergence property (i.e., $\lim_{k\to\infty}\|\nabla f(x_k)\|=0$) for nonconvex optimization by the use of the backtracking type line search in [7]. Some preliminary numerical results are reported.
Citation: Weijun Zhou, Youhua Zhou. On the strong convergence of a modified Hestenes-Stiefel method for nonconvex optimization. Journal of Industrial & Management Optimization, 2013, 9 (4) : 893-899. doi: 10.3934/jimo.2013.9.893
References:
[1]

M. R. Hestenes and E. L. Stiefel, Method of conjugate gradient for solving linear systems,, J. Res. Nat. Bur. Stand., 49 (1952), 409.   Google Scholar

[2]

D. Li and M. Fukushima, A modified BFGS method and its global convergence in nonconvex minimization,, J. Comput. Appl. Math., 129 (2001), 15.  doi: 10.1016/S0377-0427(00)00540-9.  Google Scholar

[3]

D. Li and M. Fukushima, On the global convergence of the BFGS method for nonconvex unconstrained optimization problems,, SIAM J. Optim., 11 (2001), 1054.  doi: 10.1137/S1052623499354242.  Google Scholar

[4]

J. J. Moré, B. S. Garbow and K. H. Hillstrom, Testing unconstrained optimization software,, ACM Trans. Math. Softw., 7 (1981), 17.  doi: 10.1145/355934.355936.  Google Scholar

[5]

E. Polak and G. Ribière, Note sur la convergence de méthodes de directions conjuguées,, Rev. Fr. Inform. Rech. Oper., 16 (1969), 35.   Google Scholar

[6]

B. T. Polyak, The conjugate gradient method in extreme problems,, USSR Comput. Math. Math. Phys., 9 (1969), 94.  doi: 10.1016/0041-5553(69)90035-4.  Google Scholar

[7]

L. Zhang, W. Zhou and D. Li, A descent modified Polak-Ribière-Polyak conjugate gradient method and its global convergence,, IMA J. Numer. Anal., 26 (2006), 629.  doi: 10.1093/imanum/drl016.  Google Scholar

[8]

L. Zhang, W. Zhou and D. Li, Some descent three-term conjugate gradient methods and their global convergence,, Optim. Meth. Softw., 22 (2007), 697.  doi: 10.1080/10556780701223293.  Google Scholar

[9]

W. Zhou and D. Li, On the convergence properties of the unmodified PRP method with a non-descent line search,, submitted., ().  doi: 10.1080/10556788.2013.811241.  Google Scholar

show all references

References:
[1]

M. R. Hestenes and E. L. Stiefel, Method of conjugate gradient for solving linear systems,, J. Res. Nat. Bur. Stand., 49 (1952), 409.   Google Scholar

[2]

D. Li and M. Fukushima, A modified BFGS method and its global convergence in nonconvex minimization,, J. Comput. Appl. Math., 129 (2001), 15.  doi: 10.1016/S0377-0427(00)00540-9.  Google Scholar

[3]

D. Li and M. Fukushima, On the global convergence of the BFGS method for nonconvex unconstrained optimization problems,, SIAM J. Optim., 11 (2001), 1054.  doi: 10.1137/S1052623499354242.  Google Scholar

[4]

J. J. Moré, B. S. Garbow and K. H. Hillstrom, Testing unconstrained optimization software,, ACM Trans. Math. Softw., 7 (1981), 17.  doi: 10.1145/355934.355936.  Google Scholar

[5]

E. Polak and G. Ribière, Note sur la convergence de méthodes de directions conjuguées,, Rev. Fr. Inform. Rech. Oper., 16 (1969), 35.   Google Scholar

[6]

B. T. Polyak, The conjugate gradient method in extreme problems,, USSR Comput. Math. Math. Phys., 9 (1969), 94.  doi: 10.1016/0041-5553(69)90035-4.  Google Scholar

[7]

L. Zhang, W. Zhou and D. Li, A descent modified Polak-Ribière-Polyak conjugate gradient method and its global convergence,, IMA J. Numer. Anal., 26 (2006), 629.  doi: 10.1093/imanum/drl016.  Google Scholar

[8]

L. Zhang, W. Zhou and D. Li, Some descent three-term conjugate gradient methods and their global convergence,, Optim. Meth. Softw., 22 (2007), 697.  doi: 10.1080/10556780701223293.  Google Scholar

[9]

W. Zhou and D. Li, On the convergence properties of the unmodified PRP method with a non-descent line search,, submitted., ().  doi: 10.1080/10556788.2013.811241.  Google Scholar

[1]

Thierry Horsin, Mohamed Ali Jendoubi. On the convergence to equilibria of a sequence defined by an implicit scheme. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020465

[2]

Zuliang Lu, Fei Huang, Xiankui Wu, Lin Li, Shang Liu. Convergence and quasi-optimality of $ L^2- $norms based an adaptive finite element method for nonlinear optimal control problems. Electronic Research Archive, 2020, 28 (4) : 1459-1486. doi: 10.3934/era.2020077

[3]

Parikshit Upadhyaya, Elias Jarlebring, Emanuel H. Rubensson. A density matrix approach to the convergence of the self-consistent field iteration. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 99-115. doi: 10.3934/naco.2020018

[4]

M. S. Lee, H. G. Harno, B. S. Goh, K. H. Lim. On the bang-bang control approach via a component-wise line search strategy for unconstrained optimization. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 45-61. doi: 10.3934/naco.2020014

[5]

Ahmad Z. Fino, Wenhui Chen. A global existence result for two-dimensional semilinear strongly damped wave equation with mixed nonlinearity in an exterior domain. Communications on Pure & Applied Analysis, 2020, 19 (12) : 5387-5411. doi: 10.3934/cpaa.2020243

[6]

José Madrid, João P. G. Ramos. On optimal autocorrelation inequalities on the real line. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2020271

[7]

Thomas Bartsch, Tian Xu. Strongly localized semiclassical states for nonlinear Dirac equations. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 29-60. doi: 10.3934/dcds.2020297

[8]

Awais Younus, Zoubia Dastgeer, Nudrat Ishaq, Abdul Ghaffar, Kottakkaran Sooppy Nisar, Devendra Kumar. On the observability of conformable linear time-invariant control systems. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020444

[9]

Tommi Brander, Joonas Ilmavirta, Petteri Piiroinen, Teemu Tyni. Optimal recovery of a radiating source with multiple frequencies along one line. Inverse Problems & Imaging, 2020, 14 (6) : 967-983. doi: 10.3934/ipi.2020044

[10]

Haiyu Liu, Rongmin Zhu, Yuxian Geng. Gorenstein global dimensions relative to balanced pairs. Electronic Research Archive, 2020, 28 (4) : 1563-1571. doi: 10.3934/era.2020082

[11]

Jianhua Huang, Yanbin Tang, Ming Wang. Singular support of the global attractor for a damped BBM equation. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020345

[12]

Yongge Tian, Pengyang Xie. Simultaneous optimal predictions under two seemingly unrelated linear random-effects models. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020168

[13]

Mehdi Bastani, Davod Khojasteh Salkuyeh. On the GSOR iteration method for image restoration. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 27-43. doi: 10.3934/naco.2020013

[14]

Cheng He, Changzheng Qu. Global weak solutions for the two-component Novikov equation. Electronic Research Archive, 2020, 28 (4) : 1545-1562. doi: 10.3934/era.2020081

[15]

Hong Niu, Zhijiang Feng, Qijin Xiao, Yajun Zhang. A PID control method based on optimal control strategy. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 117-126. doi: 10.3934/naco.2020019

[16]

Wenmeng Geng, Kai Tao. Large deviation theorems for dirichlet determinants of analytic quasi-periodic jacobi operators with Brjuno-Rüssmann frequency. Communications on Pure & Applied Analysis, 2020, 19 (12) : 5305-5335. doi: 10.3934/cpaa.2020240

[17]

Denis Bonheure, Silvia Cingolani, Simone Secchi. Concentration phenomena for the Schrödinger-Poisson system in $ \mathbb{R}^2 $. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020447

[18]

Mengni Li. Global regularity for a class of Monge-Ampère type equations with nonzero boundary conditions. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2020267

[19]

Bo Chen, Youde Wang. Global weak solutions for Landau-Lifshitz flows and heat flows associated to micromagnetic energy functional. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2020268

[20]

Li-Bin Liu, Ying Liang, Jian Zhang, Xiaobing Bao. A robust adaptive grid method for singularly perturbed Burger-Huxley equations. Electronic Research Archive, 2020, 28 (4) : 1439-1457. doi: 10.3934/era.2020076

2019 Impact Factor: 1.366

Metrics

  • PDF downloads (31)
  • HTML views (0)
  • Cited by (2)

Other articles
by authors

[Back to Top]