A scaled conjugate gradient method with moving asymptotes for unconstrained optimization problems

Guanghui Zhou; Qin Ni; Meilan Zeng

doi:10.3934/jimo.2016034

Article Contents

2017, Volume 13, Issue 2: 595-608. Doi: 10.3934/jimo.2016034

This issue Previous Article An fptas for the weighted number of tardy jobs minimization on a single machine with deteriorating jobs Next Article A new semi-supervised classifier based on maximum vector-angular margin

A scaled conjugate gradient method with moving asymptotes for unconstrained optimization problems

1.
Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu Province, 211106, China
2.
Huaibei Normal University, Huaibei, Anhui Province, 235000, China
3.
Hubei Engineering University, Xiaogan, Hubei Pvovince, 432000, China

Received: December 31, 2014

Revised: November 30, 2015

Published: August 2017

The authors are supported by the Natural Science Foundation of China (11471159,11571169), the Natural Science Foundation of Jiangsu Province (BK20141409), the Education Department Foundation of Anhui Province(KJ2016A651,2014jyxm161), and the Science and Technology Foundation of the Department of Education of Hubei Province (D20152701).

Abstract Full Text(HTML) Figure(6) / Table(1) Related Papers Cited by

Abstract

In this paper, a scaled method that combines the conjugate gradient with moving asymptotes is presented for solving the large-scaled nonlinear unconstrained optimization problem. A diagonal matrix is obtained by the moving asymptote technique, and a scaled gradient is determined by multiplying the gradient with the diagonal matrix. The search direction is either a scaled conjugate gradient direction or a negative scaled gradient direction under different conditions. This direction is sufficient descent if the step size satisfies the strong Wolfe condition. A global convergence analysis of this method is also provided. The numerical results show that the scaled method is efficient for solving some large-scaled nonlinear problems.

Keywords:

Mathematics Subject Classification: Primary: 65K05; Secondary: 90C30, 49M37.

Citation:

Full Text(HTML)

Figure 1. CPU time performance profiles in a log₂ scale (n = 10²)

Download: Full-size image PowerPoint slide

Figure 2. Iterations performance profiles in a log₂ scale (n = 10²)

Download: Full-size image PowerPoint slide

Figure 3. CPU time performance profiles in a log₂ scale (n = 10³)

Download: Full-size image PowerPoint slide

Figure 4. Iterations performance profiles in a log₂ scale (n = 10³)

Download: Full-size image PowerPoint slide

Figure 5. CPU time performance profiles in a log₂ scale (n = 10⁴)

Download: Full-size image PowerPoint slide

Figure 6. Iterations performance profiles in a log₂ scale (n = 10⁴)

Download: Full-size image PowerPoint slide

Table 1. Test functions

No.	Function Name	No.	Function Name
1	ARWHEAD	17	Ext quadratic penalty QP2
2	COSINE	18	Ext quadratic exponential EP1
3	EDENSCH	19	Ext Tridiagonal 2
4	EG2	20	Ext DENSCHNF
5	ENGVAL1	21	HIMMELBG
6	GENROSE	22	HIMMELH
7	Ext Freudenstein & Roth	23	TOINTGSS
8	Raydan 2	24	Extended DENSCHNB
9	Ext Tridiagonal 1	25	LIARWHD
10	Ext TET	26	Extended Trigonometric
11	Diagonal 5	27	Extended Penalty
12	Diagonal 2	28	Extended BD1
13	Ext Maratos	29	Perturbed Quadratic
14	Ext Cliff	30	Raydan 1
15	Perturbed quadratic diagonal	31	Diagonal 4
16	Ext quadratic penalty QP1	32	QUARTC

| Show Table

DownLoad: CSV

Related Papers

Cited by

References

[1]	M. Al-Baali, Y. Narushima and H. Yabe, A family of three-term conjugate gradient methods with sufficient descent property for unconstrained optimization, Computational Optimization and Applications, 60 (2015), 89-110. doi: 10.1007/s10589-014-9662-z.
[2]	N. Andrei, An unconstrained optimization test functions collection, Advanced Modeling and Optimization, 10 (2008), 147-161.
[3]	Y. Dai and Y. Yuan, A nonlinear conjugate gradient method with a strong global convergence property, SIAM Journal on Optimization, 10 (1999), 177-182. doi: 10.1137/S1052623497318992.
[4]	Y. Dai and C. Kou, A nonlinear conjugate gradient algorithm with an optimal property and an improved Wolfe line search, SIAM Journal on Optimization, 23 (2013), 296-320. doi: 10.1137/100813026.
[5]	E. D. Dolan and J. J. Moré, Benchmarking optimization software with performance profiles, Mathematical Programming, 91 (2002), 201-213. doi: 10.1007/s101070100263.
[6]	R. Fletcher and C. M. Reeves, Function minimization by conjugate gradients, The Computer Journal, 7 (1964), 149-154. doi: 10.1093/comjnl/7.2.149.
[7]	N. I. M. Gould, D. Orban and P. L. Toint, CUTEr and SifDec: A constrained and unconstrained testing environment, revisited, ACM Transactions on Mathematical Software, 29 (2003), 353-372. doi: 10.1145/962437.962438.
[8]	W. Hager and H. Zhang, A new conjugate gradient method with guaranteed descent and an efficient line search, SIAM Journal on Optimization, 16 (2005), 170-192. doi: 10.1137/030601880.
[9]	W. Hager and H. Zhang, The limited memory conjugate gradient method, SIAM Journal on Optimization, 23 (2013), 2150-2168. doi: 10.1137/120898097.
[10]	M. R. Hestenes and E. Stiefel, Methods of conjugate gradients for solving linear systems, Journal of Research of the National Bureau of Standards, 49 (1952), 409-436. doi: 10.6028/jres.049.044.
[11]	D. Luenberger and Y. Ye, Linear and Nonlinear Programming, 3^rd edition, Springer-Verlag, New York, 2008.
[12]	W. Nakamura, Y. Narushima and H. Yabe, Nonlinear conjugate gradient methods with sufficient descent properties for unconstrained optimization, Journal of Industrial and Management Optimization, 9 (2013), 595-619. doi: 10.3934/jimo.2013.9.595.
[13]	Y. Narushima, H. Yabe and J. A. Ford, A three-term conjugate gradient method with sufficient descent property for unconstrained optimization, SIAM Journal on Optimization, 21 (2011), 212-230. doi: 10.1137/080743573.
[14]	Q. Ni, A globally convergent method of moving asymptotes with trust region technique, Optimization methods and software, 18 (2003), 283-297. doi: 10.1080/1055678031000118491.
[15]	E. Polak and G. Ribiere, Note sur la convergence de méthodes de directions conjuguées, Revue française d'informatique et de recherche opérationnelle, série rouge, 3 (1969), 35-43.
[16]	B. T. Polyak, The conjugate gradient method in extremal problems, USSR Computational Mathematics and Mathematical Physics, 9 (1969), 94-112. doi: 10.1016/0041-5553(69)90035-4.
[17]	K. Svanberg, The method of moving asymptotes—a new method for structural optimization, International Journal for Numerical Methods in Engineering, 24 (2987), 359-373. doi: 10.1002/nme.1620240207.
[18]	H. Wang and Q. Ni, A new method of moving asymptotes for large-scale unconstrained optimization, Applied Mathematics and Computaiton, 203 (2008), 62-71. doi: 10.1016/j.amc.2008.03.035.
[19]	W. Zhou and Y. Zhou, On the strong convergence of a modified Hestenes-Stiefel method for nonconvex optimization, Journal of Industrial and Management Optimization, 9 (2013), 893-899. doi: 10.3934/jimo.2013.9.893.