American Institute of Mathematical Sciences

Advanced
  • Previous Article
    Convergence analysis of sparse quasi-Newton updates with positive definite matrix completion for two-dimensional functions
  • NACO Home
  • This Issue
  • Next Article
    Celis-Dennis-Tapia based approach to quadratic fractional programming problems with two quadratic constraints
2011, 1(1): 71-82. doi: 10.3934/naco.2011.1.71

A modified Fletcher-Reeves-Type derivative-free method for symmetric nonlinear equations

Dong-Hui Li 1, and  Xiao-Lin Wang 2,

1. 

School of Mathematical Sciences, South China Normal University, Guangzhou, 510631, China
2. 

College of Mathematics and Econometrics, Hunan University, Changsha, 410082, China

Received  October 2010 Revised  October 2010 Published  February 2011

In this paper, we propose a descent derivative-free method for solving symmetric nonlinear equations. The method is an extension of the modified Fletcher-Reeves (MFR) method proposed by Zhang, Zhou and Li [25] to symmetric nonlinear equations. It can be applied to solve large-scale symmetric nonlinear equations due to lower storage requirement. An attractive property of the method is that the directions generated by the method are descent for the residual function. By the use of some backtracking line search technique, the generated sequence of function values is decreasing. Under appropriate conditions, we show that the proposed method is globally convergent. The preliminary numerical results show that the method is practically effective.
Keywords: Symmetric nonlinear equations, descent direction, derivative-free method, global convergence..
Mathematics Subject Classification: Primary: 65H10; Secondary: 90C3.
Citation: Dong-Hui Li, Xiao-Lin Wang. A modified Fletcher-Reeves-Type derivative-free method for symmetric nonlinear equations. Numerical Algebra, Control & Optimization, 2011, 1 (1) : 71-82. doi: 10.3934/naco.2011.1.71
References:
[1]

M. Al-Baali, Descent property and global convergence of the Fletcher-Reeves method with inexact line search,, IMA Journal of Numerical Analysis, 5 (1985), 121. doi: 10.1093/imanum/5.1.121.

[2]

S. Bellavia and B. Morini, A globally convergent Newton-GMRES subspace method for systems of nonlinear equations,, SIAM Journal on Scientific Computing, 23 (2001), 940. doi: 10.1137/S1064827599363976.

[3]

A. Griewank, The “global” convergence of Broyden-like methods with suitable line search,, Journal of Australia Mathematical Society, 28 (1986), 75.

[4]

W. Cheng and D. H. Li, A derivative-free nonmonotone line search and its application to the spectral residual method,, IMA Journal of Numerical Analysis, 29 (2009), 814. doi: 10.1093/imanum/drn019.

[5]

Y. H. Dai and Y. Yuan, Convergence of the Fletcher-Reeves method under a generalized Wolfe search,, Journal of Computational Mathematics, 2 (1996), 142.

[6]

Y. H. Dai and Y. Yuan, Convergence properties of the Fletcher-Reeves method,, IMA Journal of Numerical Analysis, 16 (1996), 155. doi: 10.1093/imanum/16.2.155.

[7]

Y. H. Dai and Y. Yuan, "Nonlinear Conjugate Gradient Methods,", Shanghai Science and Technology Publisher, (2000).

[8]

R. Fletcher and C. Reeves, Function minimization by conjugate gradients,, Computer Journal, 7 (1964), 149. doi: 10.1093/comjnl/7.2.149.

[9]

J. C. Gilbert and J. Nocedal, Global convergence properties of conjugate gradient methods for optimization,, SIAM Journal on Optimization, 2 (1992), 21. doi: 10.1137/0802003.

[10]

G. Z. Gu, D. H. Li, L. Qi and S. Z. Zhou, Descent directions of Quasi-Newton methods for symmetric nonlinear equations,, SIAM Journal on Numerical Analysis, 40 (2003), 1763. doi: 10.1137/S0036142901397423.

[11]

J. Y. Han, G. H. Liu and H. X. Yin, Convergence properties of conjugate gradient methods with strong Wolfe linesearch,, Systems Science and Mathematical Science, 11 (1998), 112.

[12]

W. W. Hager and H. Zhang, A survey of nonlinear conjugate gradient methods,, Pacific Journal of Optimization, 2 (2006), 35.

[13]

Y. F. Hu and C. Storey, Global convergence result for conjugate gradient methods,, Journal of Optimization Theory and Applications, 71 (1991), 399. doi: 10.1007/BF00939927.

[14]

W. La Cruz and M. Raydan, Nonmonotone spectral methods for large-scale nonlinear systems,, Optimization Methods and Software, 18 (2003), 583. doi: 10.1080/10556780310001610493.

[15]

W. La Cruz, J.M. Martínez and M. Raydan, Spectral resdual method without gradient information for solving large-scale nonlinear systems of equations,, Mathematics of Computation, 75 (2006), 1429. doi: 10.1090/S0025-5718-06-01840-0.

[16]

G. H. Liu, J. Y. Han and H. X. Yin, Global convergence of the Fletcher-Reeves algorithm with an inexact line search,, Applied Mathematics, 10 (1995), 75.

[17]

D. H. Li and W. Cheng, Recent progress in the global convergence of quasi-Newton methods for nonlinear equations,, Hokkaido Journal of Mathematics, 36 (2007), 729.

[18]

D. H. Li and M. Fukushima, A globally and superlinearly convergent Gauss-Newton based BFGS method for symmetric nonlinear equations,, SIAM Journal on Numerical Analysis, 37 (1999), 152. doi: 10.1137/S0036142998335704.

[19]

D. H. Li and M. Fukushima, A derivative-free line search and global convergence of Broyden-like methods for nonlinear equations,, Optimization Methods and Software, 13 (2000), 181. doi: 10.1080/10556780008805782.

[20]

Q. Li and D. H. Li, A class of derivative-free methods for large-scale nonlinear monotone equations,, IMA Journal of Numerical Analysis, ().

[21]

M. J. D. Powell, Some convergence properties of the conjugate gradient method,, Mathematical Programming, 11 (1976), 42. doi: 10.1007/BF01580369.

[22]

M. J . D. Powell, Restart procedures of the conjugate gradient method,, Mathematical Programming, 2 (1977), 241. doi: 10.1007/BF01593790.

[23]

Q. Yan, X. Z. Peng and D. H. Li, A globally convergent derivative-free method for solving large-scale nonlinear monotone equations,, Journal of Computational and Applied Mathematics, 234 (2010), 649. doi: 10.1016/j.cam.2010.01.001.

[24]

J. Zhang and D. H. Li, A norm descent BFGS method for solving KKT systems of symmetric variational inequality problems,, Optimization Methods and Software, 22 (2007), 237. doi: 10.1080/10556780500397074.

[25]

L. Zhang, W. Zhou and D. H. Li, Global convergence of a modified Fletcher-Reeves conjugate gradient method with Armijo-type line search,, Numerische Mathematik, 104 (2006), 561. doi: 10.1007/s00211-006-0028-z.

[26]

W. Zhou and D. H. Li, Limited memory BFGS method for nonlinear monotone equations,, Journal of Computational Mathematics, 25 (2007), 89.

[27]

W. Zhou and D. H. Li, A globally convergent BFGS method for nonlinear monotone equations,, Mathematics of Computation, 77 (2008), 2231. doi: 10.1090/S0025-5718-08-02121-2.

[28]

G. Zoutendijk, Nonlinear Programming, Computational Methods,, in, (1970), 37.

show all references

References:
[1]

M. Al-Baali, Descent property and global convergence of the Fletcher-Reeves method with inexact line search,, IMA Journal of Numerical Analysis, 5 (1985), 121. doi: 10.1093/imanum/5.1.121.

[2]

S. Bellavia and B. Morini, A globally convergent Newton-GMRES subspace method for systems of nonlinear equations,, SIAM Journal on Scientific Computing, 23 (2001), 940. doi: 10.1137/S1064827599363976.

[3]

A. Griewank, The “global” convergence of Broyden-like methods with suitable line search,, Journal of Australia Mathematical Society, 28 (1986), 75.

[4]

W. Cheng and D. H. Li, A derivative-free nonmonotone line search and its application to the spectral residual method,, IMA Journal of Numerical Analysis, 29 (2009), 814. doi: 10.1093/imanum/drn019.

[5]

Y. H. Dai and Y. Yuan, Convergence of the Fletcher-Reeves method under a generalized Wolfe search,, Journal of Computational Mathematics, 2 (1996), 142.

[6]

Y. H. Dai and Y. Yuan, Convergence properties of the Fletcher-Reeves method,, IMA Journal of Numerical Analysis, 16 (1996), 155. doi: 10.1093/imanum/16.2.155.

[7]

Y. H. Dai and Y. Yuan, "Nonlinear Conjugate Gradient Methods,", Shanghai Science and Technology Publisher, (2000).

[8]

R. Fletcher and C. Reeves, Function minimization by conjugate gradients,, Computer Journal, 7 (1964), 149. doi: 10.1093/comjnl/7.2.149.

[9]

J. C. Gilbert and J. Nocedal, Global convergence properties of conjugate gradient methods for optimization,, SIAM Journal on Optimization, 2 (1992), 21. doi: 10.1137/0802003.

[10]

G. Z. Gu, D. H. Li, L. Qi and S. Z. Zhou, Descent directions of Quasi-Newton methods for symmetric nonlinear equations,, SIAM Journal on Numerical Analysis, 40 (2003), 1763. doi: 10.1137/S0036142901397423.

[11]

J. Y. Han, G. H. Liu and H. X. Yin, Convergence properties of conjugate gradient methods with strong Wolfe linesearch,, Systems Science and Mathematical Science, 11 (1998), 112.

[12]

W. W. Hager and H. Zhang, A survey of nonlinear conjugate gradient methods,, Pacific Journal of Optimization, 2 (2006), 35.

[13]

Y. F. Hu and C. Storey, Global convergence result for conjugate gradient methods,, Journal of Optimization Theory and Applications, 71 (1991), 399. doi: 10.1007/BF00939927.

[14]

W. La Cruz and M. Raydan, Nonmonotone spectral methods for large-scale nonlinear systems,, Optimization Methods and Software, 18 (2003), 583. doi: 10.1080/10556780310001610493.

[15]

W. La Cruz, J.M. Martínez and M. Raydan, Spectral resdual method without gradient information for solving large-scale nonlinear systems of equations,, Mathematics of Computation, 75 (2006), 1429. doi: 10.1090/S0025-5718-06-01840-0.

[16]

G. H. Liu, J. Y. Han and H. X. Yin, Global convergence of the Fletcher-Reeves algorithm with an inexact line search,, Applied Mathematics, 10 (1995), 75.

[17]

D. H. Li and W. Cheng, Recent progress in the global convergence of quasi-Newton methods for nonlinear equations,, Hokkaido Journal of Mathematics, 36 (2007), 729.

[18]

D. H. Li and M. Fukushima, A globally and superlinearly convergent Gauss-Newton based BFGS method for symmetric nonlinear equations,, SIAM Journal on Numerical Analysis, 37 (1999), 152. doi: 10.1137/S0036142998335704.

[19]

D. H. Li and M. Fukushima, A derivative-free line search and global convergence of Broyden-like methods for nonlinear equations,, Optimization Methods and Software, 13 (2000), 181. doi: 10.1080/10556780008805782.

[20]

Q. Li and D. H. Li, A class of derivative-free methods for large-scale nonlinear monotone equations,, IMA Journal of Numerical Analysis, ().

[21]

M. J. D. Powell, Some convergence properties of the conjugate gradient method,, Mathematical Programming, 11 (1976), 42. doi: 10.1007/BF01580369.

[22]

M. J . D. Powell, Restart procedures of the conjugate gradient method,, Mathematical Programming, 2 (1977), 241. doi: 10.1007/BF01593790.

[23]

Q. Yan, X. Z. Peng and D. H. Li, A globally convergent derivative-free method for solving large-scale nonlinear monotone equations,, Journal of Computational and Applied Mathematics, 234 (2010), 649. doi: 10.1016/j.cam.2010.01.001.

[24]

J. Zhang and D. H. Li, A norm descent BFGS method for solving KKT systems of symmetric variational inequality problems,, Optimization Methods and Software, 22 (2007), 237. doi: 10.1080/10556780500397074.

[25]

L. Zhang, W. Zhou and D. H. Li, Global convergence of a modified Fletcher-Reeves conjugate gradient method with Armijo-type line search,, Numerische Mathematik, 104 (2006), 561. doi: 10.1007/s00211-006-0028-z.

[26]

W. Zhou and D. H. Li, Limited memory BFGS method for nonlinear monotone equations,, Journal of Computational Mathematics, 25 (2007), 89.

[27]

W. Zhou and D. H. Li, A globally convergent BFGS method for nonlinear monotone equations,, Mathematics of Computation, 77 (2008), 2231. doi: 10.1090/S0025-5718-08-02121-2.

[28]

G. Zoutendijk, Nonlinear Programming, Computational Methods,, in, (1970), 37.

[1]

Wei-Zhe Gu, Li-Yong Lu. The linear convergence of a derivative-free descent method for nonlinear complementarity problems. Journal of Industrial & Management Optimization, 2017, 13 (2) : 531-548. doi: 10.3934/jimo.2016030
[2]

Gaohang Yu. A derivative-free method for solving large-scale nonlinear systems of equations. Journal of Industrial & Management Optimization, 2010, 6 (1) : 149-160. doi: 10.3934/jimo.2010.6.149
[3]

A. M. Bagirov, Moumita Ghosh, Dean Webb. A derivative-free method for linearly constrained nonsmooth optimization. Journal of Industrial & Management Optimization, 2006, 2 (3) : 319-338. doi: 10.3934/jimo.2006.2.319
[4]

Liang Zhang, Wenyu Sun, Raimundo J. B. de Sampaio, Jinyun Yuan. A wedge trust region method with self-correcting geometry for derivative-free optimization. Numerical Algebra, Control & Optimization, 2015, 5 (2) : 169-184. doi: 10.3934/naco.2015.5.169
[5]

Herbert Gajewski, Jens A. Griepentrog. A descent method for the free energy of multicomponent systems. Discrete & Continuous Dynamical Systems - A, 2006, 15 (2) : 505-528. doi: 10.3934/dcds.2006.15.505
[6]

Jun Takaki, Nobuo Yamashita. A derivative-free trust-region algorithm for unconstrained optimization with controlled error. Numerical Algebra, Control & Optimization, 2011, 1 (1) : 117-145. doi: 10.3934/naco.2011.1.117
[7]

Ugur G. Abdulla. On the optimal control of the free boundary problems for the second order parabolic equations. II. Convergence of the method of finite differences. Inverse Problems & Imaging, 2016, 10 (4) : 869-898. doi: 10.3934/ipi.2016025
[8]

Gaohang Yu, Lutai Guan, Guoyin Li. Global convergence of modified Polak-Ribière-Polyak conjugate gradient methods with sufficient descent property. Journal of Industrial & Management Optimization, 2008, 4 (3) : 565-579. doi: 10.3934/jimo.2008.4.565
[9]

Ugur G. Abdulla. On the optimal control of the free boundary problems for the second order parabolic equations. I. Well-posedness and convergence of the method of lines. Inverse Problems & Imaging, 2013, 7 (2) : 307-340. doi: 10.3934/ipi.2013.7.307
[10]

M. S. Lee, B. S. Goh, H. G. Harno, K. H. Lim. On a two-phase approximate greatest descent method for nonlinear optimization with equality constraints. Numerical Algebra, Control & Optimization, 2018, 8 (3) : 315-326. doi: 10.3934/naco.2018020
[11]

Hideo Takaoka. Energy transfer model for the derivative nonlinear Schrödinger equations on the torus. Discrete & Continuous Dynamical Systems - A, 2017, 37 (11) : 5819-5841. doi: 10.3934/dcds.2017253
[12]

Nakao Hayashi, Pavel I. Naumkin, Patrick-Nicolas Pipolo. Smoothing effects for some derivative nonlinear Schrödinger equations. Discrete & Continuous Dynamical Systems - A, 1999, 5 (3) : 685-695. doi: 10.3934/dcds.1999.5.685
[13]

Paola Mannucci. The Dirichlet problem for fully nonlinear elliptic equations non-degenerate in a fixed direction. Communications on Pure & Applied Analysis, 2014, 13 (1) : 119-133. doi: 10.3934/cpaa.2014.13.119
[14]

Aimin Huang, Roger Temam. The nonlinear 2D subcritical inviscid shallow water equations with periodicity in one direction. Communications on Pure & Applied Analysis, 2014, 13 (5) : 2005-2038. doi: 10.3934/cpaa.2014.13.2005
[15]

Zhili Ge, Gang Qian, Deren Han. Global convergence of an inexact operator splitting method for monotone variational inequalities. Journal of Industrial & Management Optimization, 2011, 7 (4) : 1013-1026. doi: 10.3934/jimo.2011.7.1013
[16]

Liyan Qi, Xiantao Xiao, Liwei Zhang. On the global convergence of a parameter-adjusting Levenberg-Marquardt method. Numerical Algebra, Control & Optimization, 2015, 5 (1) : 25-36. doi: 10.3934/naco.2015.5.25
[17]

Flavia Smarrazzo. On a class of equations with variable parabolicity direction. Discrete & Continuous Dynamical Systems - A, 2008, 22 (3) : 729-758. doi: 10.3934/dcds.2008.22.729
[18]

Tahereh Salimi Siahkolaei, Davod Khojasteh Salkuyeh. A preconditioned SSOR iteration method for solving complex symmetric system of linear equations. Numerical Algebra, Control & Optimization, 2019, 0 (0) : 0-0. doi: 10.3934/naco.2019033
[19]

Russell E. Warren, Stanley J. Osher. Hyperspectral unmixing by the alternating direction method of multipliers. Inverse Problems & Imaging, 2015, 9 (3) : 917-933. doi: 10.3934/ipi.2015.9.917
[20]

Zihua Guo, Yifei Wu. Global well-posedness for the derivative nonlinear Schrödinger equation in $H^{\frac 12} (\mathbb{R} )$. Discrete & Continuous Dynamical Systems - A, 2017, 37 (1) : 257-264. doi: 10.3934/dcds.2017010

 Impact Factor: 

Tools

Metrics

  • PDF downloads (14)
  • HTML views (0)
  • Cited by (16)

Other articles
by authors

[Back to Top]

Copyright © 2019 American Institute of Mathematical Sciences