The bundle scheme for solving arbitrary eigenvalue optimizations

Ming Huang; Xi-Jun Liang; Yuan Lu; Li-Ping Pang

doi:10.3934/jimo.2016039

Article Contents

2017, Volume 13, Issue 2: 659-680. Doi: 10.3934/jimo.2016039

This issue Previous Article A class of descent four–term extension of the Dai–Liao conjugate gradient method based on the scaled memoryless BFGS update Next Article Algorithms for single-machine scheduling problem with deterioration depending on a novel model

The bundle scheme for solving arbitrary eigenvalue optimizations

1.
Department of Mathematics, Dalian Maritime University, Dalian 116026, China
2.
School of Control Science and Engineering, Dalian University of Technology, Dalian 116024, China
3.
College of Science, China University of Petroleum, Qingdao 266580, China
4.
School of Sciences, Shenyang University, Shenyang 110044, China
5.
School of Mathematical Sciences, Dalian University of Technology, Dalian 116024, China

* Corresponding author
* Corresponding author

Received: March 31, 2015

Revised: December 31, 2015

Published: August 2017

The first and forth authors' work was supported in part by the National Natural Science Foundation of China under projects No.11171049. The first author was supported in part by National Natural Science Foundation of China under projects No.11626053, the Project funded by China Postdoctoral Science Fundation under No. 2016M601296, Fundamental Research Funds for the Central Universities under projects No.3132016108 and Scientific Research Foundation Funds of DLMU under projects No.02501102. The second author' work was supported in part by the National Natural Science Foundation of China under projects No. 61503412 and Natural Science Foundation of Shandong Province under Grant No. ZR2014AP004. The third author' work was supported in part by the National Natural Science Foundation of China under projects No.11301347, and Program for Liaoning Excellent Talents in University under projects No. LJQ2015075.

Abstract Full Text(HTML) Figure(0) / Table(2) Related Papers Cited by

Abstract

Optimization involving eigenvalues arises in a large spectrum of applications in various domains, such as physics, engineering, statistics and finance. In this paper, we consider the arbitrary eigenvalue minimization problems over an affine family of symmetric matrices, which is a special class of eigenvalue function--D.C. function $λ^_{l}$ . An explicit proximal bundle approach for solving this class of nonsmooth, nonconvex (D.C.) optimization problem is developed. We prove the global convergence of our method, in the sense that every accumulation point of the sequence of iterates generated by the proposed algorithm is stationary. As an application, we use the proposed bundle method to solve some particular eigenvalue problems. Some encouraging preliminary numerical results indicate that our method can solve the test problems efficiently.

Keywords:

Mathematics Subject Classification: Primary: 49J52, 15A18; Secondary: 90C26, 65K10.

Citation:

Full Text(HTML)

Table 1. Numerical results for the second largest eigenvalue

n	$n_n$	$n_d$	$\#f/g$	$x^k$	Time	$fx^k$	Res
2	11	22	252	$1.0e-06 *(0.4508,0.0685)$	0.6093	2.8348e-07	4.3320e-07
3	15	25	308	$1.0e-06 *(0.2321,0.3796)$	0.8073	1.9516e-07	2.9823e-07
4	11	24	270	$1.0e-06 *(0.3036,0.4796)$	0.7225	2.4712e-07	3.7764e-07
5	17	21	288	$1.0e-06 *(0.4569,0.7587)$	0.8452	3.8994e-07	5.9589e-07
6	12	23	276	$1.0e-06 *(0.3141,0.2245)$	0.6242	2.0441e-07	3.1237e-07
7	27	24	266	$1.0e-06 *(0.3944,0.0098)$	0.8933	2.2090e-07	3.3757e-07
8	15	23	260	$1.0e-06 *(0.3177,0.5355)$	0.6810	2.7525e-07	4.2062e-07
9	13	24	284	$1.0e-06 *(0.4139,0.0045)$	0.7899	2.2820e-07	3.4873e-07
10	15	22	302	$1.0e-06 *(0.0770,0.2213)$	0.7927	1.2711e-07	1.9425e-07
11	16	26	256	$1.0e-06 *(0.1415,0.1247)$	0.9039	9.1817e-08	1.4031e-07
12	15	24	274	$1.0e-06 *(0.3637,0.0449)$	0.6943	2.2398e-07	3.4228e-07
13	14	23	270	$1.0e-06 *(0.3033,0.4745)$	0.7462	2.4475e-07	3.7402e-07
14	18	25	292	$1.0e-06 *(0.1657,0.1121)$	0.9307	1.0812e-07	1.6522e-07
15	15	22	268	$1.0e-06 *(0.4380,0.0246)$	0.7305	2.5366e-07	3.8764e-07
16	13	24	284	$1.0e-06 *(0.0836,0.2209)$	0.7983	1.2407e-07	1.8960e-07
17	14	23	270	$1.0e-06 *(0.3016,0.4746)$	0.7458	2.4466e-07	3.7388e-07
18	22	16	242	$1.0e-06 *(0.5804,0.7083)$	0.5951	3.9904e-07	6.0980e-07
19	28	23	354	$1.0e-06 *(0.5475,0.7792)$	1.1864	4.0963e-07	6.2599e-07
20	17	24	288	$1.0e-06 *(0.3219,0.5632)$	0.7114	2.8989e-07	4.4300e-07

| Show Table

DownLoad: CSV

Table 2. Comparison results of Algorithm 4.1 and Generalized cutting plane method (GCPM)

Algorithm 4.1				GCPM
n	$\#f/g$	Time	Res	$\#f/g$	Time	Res
5	214	0.6762	4.7140e-06	338	1.4614	4.7709e-06
10	246	0.8603	4.4696e-06	308	1.2572	4.8573e-06
15	234	0.7433	3.8621e-06	420	1.5076	4.8546e-06
20	266	0.8157	4.2591e-06	418	1.4537	4.6413e-06
25	235	0.7141	3.8623e-06	398	1.4743	4.7922e-06
30	290	0.7934	4.2803e-06	406	1.3006	5.4616e-06
35	258	0.7645	4.1979e-06	414	1.2542	4.8415e-06
40	290	0.8635	4.4795e-06	398	1.3216	5.8507e-06
45	238	0.7330	3.9895e-06	566	1.9189	4.5721e-06
50	282	0.8268	4.5536e-06	532	1.5479	6.6442e-06

| Show Table

DownLoad: CSV

Related Papers

Cited by

References

[1]	F. Alizadeh, J.-P. A. Haeberly and M. L. Overton, Primal-dual interior-point methods for semidefinite programming: Convergence rates, stability and numerical results, SIAM Journal on Optimization, 8 (1998), 746-768. doi: 10.1137/S1052623496304700.
[2]	F. Alizadeh, Interior point methods in semidefinite programming with applications to combinatorial optimization, SIAM Journal on Optimization, 5 (1995), 13-51. doi: 10.1137/0805002.
[3]	R. Bhatia, Matrix Analysis, Graduate Texts in Mathematics, Vol. 169, Springer-Verlag, New York, 1997. doi: 10.1007/978-1-4612-0653-8.
[4]	F. H. Clarke, Optimization and Nonsmooth Analysis, Classics in Applied Mathematics, Vol. 5, Society for Industry and Applied Mathematics (SIAM), Philadelphia, 1990. doi: 10.1137/1.9781611971309.
[5]	F. H. Clarke, Y. S. Ledyaev, R. J. Stern and P. R. Wolenski, Nonsmooth Analysis and Control Theory, Graduate Texts in Mathematics, Vol. 178, Springer-Verlag, New York, 1998. doi: 10.1007/b97650.
[6]	J. Cullum, W. E. Donath and P. Wolfe, The minimization of certain nondifferentiable sums of eigenvalues of symmetric matrices, Mathematical Programming Study, 3 (1975), 35-55. doi: 10.1007/BFb0120698.
[7]	K. Fan, On a theorem of Weyl concerning eigenvalues of linear transformations. Ⅰ., Proc. Nat. Acad. Sci. U.S.A., 35 (1949), 652-655. doi: 10.1073/pnas.35.11.652.
[8]	S. Friedland, J. Nocedal and M. L. Overton, The formulation and analysis of numerical methods for inverse eigenvalue problems, SIAM Journal on Numerical Analysis, 24 (1987), 634-667. doi: 10.1137/0724043.
[9]	M. Huang, L. P. Pang and Z. Q. Xia, The space decomposition theory for a class of eigenvalue optimizations, Computational Optimization and Applications, 58 (2014), 423-454. doi: 10.1007/s10589-013-9624-x.
[10]	C. Helmberg and F. Rendl, A spectral bundle method for semidefinite programming, SIAM Journal on Optimization, 10 (2000), 673-696. doi: 10.1137/S1052623497328987.
[11]	J. -B. Hiriart-Urruty and C. Lemaréchal, Convex Analysis and Minimization Algorithms Ⅰ-Ⅱ, Grundlehren der mathematischen Wissenschaften, Vols 305-306, Springer-Verlag, Berlin, 1993. doi: 10.1007/978-3-662-02796-7.
[12]	J.-B. Hiriart-Urruty and D. Ye, Sensitivity analysis of all eigenvalues of a symmetric matrix, Numerische Mathematik, 70 (1995), 45-72. doi: 10.1007/s002110050109.
[13]	F. Jarre, An interior-point method for minimizing the maximum eigenvalue of a linear combination of matrices, SIAM Journal on Control and Optimization, 31 (1993), 1360-1377. doi: 10.1137/0331064.
[14]	M. Kojima, M. Shida and S. Shindoh, Local convergence of predictor-corrector infeasible-interior-point algorithms for SDPs and SDLCPs, Mathematical Programming, 80 (1998), 129-160. doi: 10.1007/BF01581723.
[15]	K. C. Kiwiel, An aggregate subgradient method for nonsmooth convex minimization, Mathematical Programming, 27 (1983), 320-341. doi: 10.1007/BF02591907.
[16]	K. C. Kiwiel, A linearization algorithm for nonsmooth minimization, Mathematics of Operations Research, 10 (1985), 185-194. doi: 10.1287/moor.10.2.185.
[17]	K. C. Kiwiel, Proximity control in bundle methods for convex nondifferentiable minimization, Mathematical Programming, 46 (1990), 105-122. doi: 10.1007/BF01585731.
[18]	C. Lemaréchal, An extension of davidon methods to nondifferentiable problems, Mathematical Programming Study, 3 (2009), 95-109. doi: 10.1007/BFb0120700.
[19]	A. S. Lewis and M. L. Overton, Eigenvalue optimization, Acta Numerica, 5 (1996), 149-190. doi: 10.1017/S0962492900002646.
[20]	R. Mifflin, An algorithm for constrained optimization with semismooth functions, Mathematics of Operations Research, 2 (1977), 191-207. doi: 10.1287/moor.2.2.191.
[21]	Y. Nesterov, Interior-point methods: An old and new approach to nonlinear programming, Mathematical Programming, 79 (1997), 285-297. doi: 10.1007/BF02614321.
[22]	Y. Nesterov and A. Nemirovskii, A General Approach to Polynomial-Time Algorithms Design for Convex Programming, Technical report, Centr. Econ. & Math. Inst., USSR Academy of Sciences, Moscow, USSR, 1988.
[23]	Y. Nesterov and A. Nemirovskii, Interior-Point Polynomial Algorithms in Convex Programming, SIAM Studies in Applied and Numerical Mathematics, Vol. 13, Philadelphia, 1994. doi: 10.1137/1.9781611970791.
[24]	F. Oustry, The ${\mathcal U}$-Lagrangian of the maximum eigenvalue function, SIAM Journal on Optimization, 9 (1999), 526-549. doi: 10.1137/S1052623496311776.
[25]	M. L. Overton, On minimizing the maximum eigenvalue of a symmetric matrix, SIAM Journal on Matrix Analysis and Applications, 9 (1988), 256-268. doi: 10.1137/0609021.
[26]	M. L. Overton and R. S. Womersley, Optimality conditions and duality theory for minimizing sums of the largest eigenvalues of symmetric matrices, Mathematical Programming, 62 (1993), 321-357. doi: 10.1007/BF01585173.
[27]	G. Pataki, On the rank of extreme matrices in semidefinite programs and the multiplicity of optimal eigenvalues, Mathematics of Operations Research, 23 (1998), 339-358. doi: 10.1287/moor.23.2.339.
[28]	E. Polak, On the mathematical foundations of nondifferentiable optimization in engineering design, SIAM Review, 29 (1987), 21-89. doi: 10.1137/1029002.
[29]	E. Polak and Y. Wardi, Nondifferentiable optimization algorithm for designing control systems having singular value inequalities, Automatica, 18 (1982), 267-283. doi: 10.1016/0005-1098(82)90087-5.
[30]	S. B. Robinson, On the second eigenvalue for nonhomogeneous quasi-linear operators, SIAM Journal on Mathematical Analysis, 35 (2004), 1241-1249. doi: 10.1137/S0036141003426008.
[31]	R. T. Rockafellar, Convex Analysis, Princeton University Press, NJ, 1997.
[32]	A. Shapiro and M. K. H. Fan, On eigenvalue optimization, SIAM Journal on Optimization, 5 (1995), 552-569. doi: 10.1137/0805028.
[33]	A. Shapiro, First and second order analysis of nonlinear semidefinite programs, Mathematical Programming, 77 (1997), 301-320. doi: 10.1007/BF02614439.
[34]	J. Sun, S. Boyd, L. Xiao and P. Diaconis, The fastest mixing markov process on a graph and a connection to a maximum variance unfolding problem, SIAM Review, 48 (2006), 681-699. doi: 10.1137/S0036144504443821.
[35]	M. Torki, First-and second-order epi-differentiability in eigenvalue optimization, Journal of Mathematical Analysis and Applications, 234 (1999), 391-416. doi: 10.1006/jmaa.1999.6320.
[36]	M. Torki, Second-order directional derivatives of all eigenvalues of a symmetric matrix, Nonlinear Analysis. Theory, Methods & Applications, 46 (2001), 1133-1150. doi: 10.1016/S0362-546X(00)00165-6.
[37]	L. Vandenberghe and S. Boyd, Semidefinite programming, SIAM Review, 38 (1996), 49-95. doi: 10.1137/1038003.
[38]	J. Vlček and L. Lukšan, Globally convergent variable metric method for nonconvex nondifferentiable unconstrained minimization, Journal of Optimization Theory and Applications, 111 (2001), 407-430. doi: 10.1023/A:1011990503369.
[39]	P. Wolfe, A method of conjugate subgradients for minimizing nondifferentiable functions, Mathematical Programming Study, 3 (1975), 145-173. doi: 10.1007/BFb0120703.