September  2018, 8(3): 377-387. doi: 10.3934/naco.2018024

Multi-step spectral gradient methods with modified weak secant relation for large scale unconstrained optimization

1. 

Department of Mathematical and Actuarial Sciences, Lee Kong Chian Faculty of Engineering and Science, Universiti Tunku Abdul Rahman, Sungai Long Campus, Jalan Sungai Long 9, Bandar Sungai Long, 43000 Kajang, Selangor, Malaysia

2. 

Department of Mathematics, Faculty of Science, Universiti Putra Malaysia, 43400 Serdang, Selangor, Malaysia

3. 

Institute for Mathematical Research, Universiti Putra Malaysia, 43400 Serdang, Selangor, Malaysia

* Corresponding author: Hong Seng Sim

Received  May 2017 Revised  March 2018 Published  June 2018

Fund Project: The first author is supported by Yayasan Sultan Iskandar Johor 2014.

In this paper, we aim to propose some spectral gradient methods via variational technique under log-determinant norm. The spectral parameters satisfy the modified weak secant relations that inspired by the multistep approximation for solving large scale unconstrained optimization. An executable code is developed to test the efficiency of the proposed method with spectral gradient method using standard weak secant relation as constraint. Numerical results are presented which suggest a better performance has been achieved.

Citation: Hong Seng Sim, Wah June Leong, Chuei Yee Chen, Siti Nur Iqmal Ibrahim. Multi-step spectral gradient methods with modified weak secant relation for large scale unconstrained optimization. Numerical Algebra, Control & Optimization, 2018, 8 (3) : 377-387. doi: 10.3934/naco.2018024
References:
[1]

N. Andrei, An unconstrained optimization test functions collection, Advanced Modeling and Optimization, 10 (2008), 147-161.   Google Scholar

[2]

J. Barzilai and J. Borwein, Two-point step size gradient methods, IMA Journal of Numerical Analysis, 8 (1988), 141-148.  doi: 10.1093/imanum/8.1.141.  Google Scholar

[3]

I. BongartzA. R. ConnN. I. M. Gould and Ph. L. Toint, CUTE: constrained and unconstrained testing environment, ACM Transactions on Mathematical Software, 21 (1995), 123-160.  doi: 10.1145/200979.201043.  Google Scholar

[4]

R. H. Byrd and J. Nocedal, A tool for the analysis of quasi-Newton methods with application to unconstrained minimization, SIAM J. Numer. Anal., 26 (1989), 727-739.  doi: 10.1137/0726042.  Google Scholar

[5]

J. E. Dennis and H. Wolkowicz, Sizing and least change secant methods, SIAM Journal on Numerical Analysis, 30 (1993), 1291-1313.  doi: 10.1137/0730067.  Google Scholar

[6]

J. A. Ford and I. A. Moghrabi, Multi-step quasi-Newton methods for optimization, Journal of Computational and Applied Mathematics, 50 (1994), 305-323.  doi: 10.1016/0377-0427(94)90309-3.  Google Scholar

[7]

W. La CruzJ. Martinez and M. Raydan, Spectral residual method without gradient information for solving large-scale nonlinear systems of equations, Math. Comput., 75 (2006), 1449-1466.  doi: 10.1090/S0025-5718-06-01840-0.  Google Scholar

[8]

W. La Cruz and M. Raydan, Nonmonotone spectral methods for large-scale nonlinear systems, Optim. Methods Softw., 18 (2003), 583-599.  doi: 10.1080/10556780310001610493.  Google Scholar

[9]

D. C. Liu and J. Nocedal, On the limited memory BFGS method for large scale optimization, Mathematical Programming, 45 (1989), 503-528.  doi: 10.1007/BF01589116.  Google Scholar

[10]

M. Raydan, The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem, SIAM Journal of Optimization, 7 (1997), 26-33.  doi: 10.1137/S1052623494266365.  Google Scholar

[11]

M. ZhuJ. L. Nazareth and H. Wolkowicz, The quasi-Cauchy relation and diagonal updating, SIAM Journal on Optimization, 4 (1999), 1192-1204.  doi: 10.1137/S1052623498331793.  Google Scholar

show all references

References:
[1]

N. Andrei, An unconstrained optimization test functions collection, Advanced Modeling and Optimization, 10 (2008), 147-161.   Google Scholar

[2]

J. Barzilai and J. Borwein, Two-point step size gradient methods, IMA Journal of Numerical Analysis, 8 (1988), 141-148.  doi: 10.1093/imanum/8.1.141.  Google Scholar

[3]

I. BongartzA. R. ConnN. I. M. Gould and Ph. L. Toint, CUTE: constrained and unconstrained testing environment, ACM Transactions on Mathematical Software, 21 (1995), 123-160.  doi: 10.1145/200979.201043.  Google Scholar

[4]

R. H. Byrd and J. Nocedal, A tool for the analysis of quasi-Newton methods with application to unconstrained minimization, SIAM J. Numer. Anal., 26 (1989), 727-739.  doi: 10.1137/0726042.  Google Scholar

[5]

J. E. Dennis and H. Wolkowicz, Sizing and least change secant methods, SIAM Journal on Numerical Analysis, 30 (1993), 1291-1313.  doi: 10.1137/0730067.  Google Scholar

[6]

J. A. Ford and I. A. Moghrabi, Multi-step quasi-Newton methods for optimization, Journal of Computational and Applied Mathematics, 50 (1994), 305-323.  doi: 10.1016/0377-0427(94)90309-3.  Google Scholar

[7]

W. La CruzJ. Martinez and M. Raydan, Spectral residual method without gradient information for solving large-scale nonlinear systems of equations, Math. Comput., 75 (2006), 1449-1466.  doi: 10.1090/S0025-5718-06-01840-0.  Google Scholar

[8]

W. La Cruz and M. Raydan, Nonmonotone spectral methods for large-scale nonlinear systems, Optim. Methods Softw., 18 (2003), 583-599.  doi: 10.1080/10556780310001610493.  Google Scholar

[9]

D. C. Liu and J. Nocedal, On the limited memory BFGS method for large scale optimization, Mathematical Programming, 45 (1989), 503-528.  doi: 10.1007/BF01589116.  Google Scholar

[10]

M. Raydan, The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem, SIAM Journal of Optimization, 7 (1997), 26-33.  doi: 10.1137/S1052623494266365.  Google Scholar

[11]

M. ZhuJ. L. Nazareth and H. Wolkowicz, The quasi-Cauchy relation and diagonal updating, SIAM Journal on Optimization, 4 (1999), 1192-1204.  doi: 10.1137/S1052623498331793.  Google Scholar

Figure 1.  Performance Profiling for the Modified Multiple Spectral Gradient Methods and Standard Multiple Spectral Gradient Method in terms of Number of Iterations.
Figure 2.  Performance Profiling for the Modified Multiple Spectral Gradient Methods and Standard Multiple Spectral Gradient Method in terms of Number of Function Calls.
Figure 3.  Performance Profiling for the Modified Multiple Spectral Gradient Methods and Standard Multiple Spectral Gradient Method in terms of CPU Time.
[1]

Predrag S. Stanimirović, Branislav Ivanov, Haifeng Ma, Dijana Mosić. A survey of gradient methods for solving nonlinear optimization. Electronic Research Archive, 2020, 28 (4) : 1573-1624. doi: 10.3934/era.2020115

[2]

M. S. Lee, H. G. Harno, B. S. Goh, K. H. Lim. On the bang-bang control approach via a component-wise line search strategy for unconstrained optimization. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 45-61. doi: 10.3934/naco.2020014

[3]

Huiying Fan, Tao Ma. Parabolic equations involving Laguerre operators and weighted mixed-norm estimates. Communications on Pure & Applied Analysis, 2020, 19 (12) : 5487-5508. doi: 10.3934/cpaa.2020249

[4]

Guido Cavallaro, Roberto Garra, Carlo Marchioro. Long time localization of modified surface quasi-geostrophic equations. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020336

[5]

Xuefei He, Kun Wang, Liwei Xu. Efficient finite difference methods for the nonlinear Helmholtz equation in Kerr medium. Electronic Research Archive, 2020, 28 (4) : 1503-1528. doi: 10.3934/era.2020079

[6]

Cheng He, Changzheng Qu. Global weak solutions for the two-component Novikov equation. Electronic Research Archive, 2020, 28 (4) : 1545-1562. doi: 10.3934/era.2020081

[7]

Alberto Bressan, Sondre Tesdal Galtung. A 2-dimensional shape optimization problem for tree branches. Networks & Heterogeneous Media, 2020  doi: 10.3934/nhm.2020031

[8]

Andreu Ferré Moragues. Properties of multicorrelation sequences and large returns under some ergodicity assumptions. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020386

[9]

Xin Guo, Lei Shi. Preface of the special issue on analysis in data science: Methods and applications. Mathematical Foundations of Computing, 2020, 3 (4) : i-ii. doi: 10.3934/mfc.2020026

[10]

Antoine Benoit. Weak well-posedness of hyperbolic boundary value problems in a strip: when instabilities do not reflect the geometry. Communications on Pure & Applied Analysis, 2020, 19 (12) : 5475-5486. doi: 10.3934/cpaa.2020248

[11]

Wenmeng Geng, Kai Tao. Large deviation theorems for dirichlet determinants of analytic quasi-periodic jacobi operators with Brjuno-Rüssmann frequency. Communications on Pure & Applied Analysis, 2020, 19 (12) : 5305-5335. doi: 10.3934/cpaa.2020240

[12]

Bo Chen, Youde Wang. Global weak solutions for Landau-Lifshitz flows and heat flows associated to micromagnetic energy functional. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2020268

[13]

Shenglan Xie, Maoan Han, Peng Zhu. A posteriori error estimate of weak Galerkin fem for second order elliptic problem with mixed boundary condition. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020340

[14]

Annegret Glitzky, Matthias Liero, Grigor Nika. Dimension reduction of thermistor models for large-area organic light-emitting diodes. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020460

[15]

Yongxiu Shi, Haitao Wan. Refined asymptotic behavior and uniqueness of large solutions to a quasilinear elliptic equation in a borderline case. Electronic Research Archive, , () : -. doi: 10.3934/era.2020119

[16]

Wenbin Li, Jianliang Qian. Simultaneously recovering both domain and varying density in inverse gravimetry by efficient level-set methods. Inverse Problems & Imaging, , () : -. doi: 10.3934/ipi.2020073

[17]

Suhaib Abduljabbar Altamir, Zakariya Yahya Algamal. Improving whale optimization algorithm for feature selection with a time-varying transfer function. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 87-98. doi: 10.3934/naco.2020017

 Impact Factor: 

Metrics

  • PDF downloads (145)
  • HTML views (241)
  • Cited by (0)

[Back to Top]