Homotopy method for matrix rank minimization based on the matrix hard thresholding method

Zhengshan Dong; Jianli Chen; Wenxing Zhu

doi:10.3934/naco.2019015

Article Contents

2019, Volume 9, Issue 2: 211-224. Doi: 10.3934/naco.2019015

This issue Previous Article Application of robust optimization for a product portfolio problem using an invasive weed optimization algorithm Next Article Indirect methods for fuel-minimal rendezvous with a large population of temporarily captured orbiters

Homotopy method for matrix rank minimization based on the matrix hard thresholding method

1.
Ruijie Networks Co., Ltd, Fuzhou 350108, China
2.
Center for Discrete Mathematics and Theoretical Computer Science, Fuzhou University, Fuzhou 350108, China

^* Corresponding author: wxzhu@fzu.edu.cn
^* Corresponding author: wxzhu@fzu.edu.cn

Received: May 2018

Revised: August 2018

Published: January 2019

This research is supported by the National Natural Science Foundation of China under Grant 61672005

Abstract Full Text(HTML) Figure(0) / Table(5) Related Papers Cited by

Abstract

Based on the matrix hard thresholding method, a homotopy method is proposed for solving the matrix rank minimization problem. This method iteratively solves a series of regularization subproblems, whose solutions are given in closed form by the matrix hard thresholding operator. Under some mild assumptions, convergence of the proposed method is proved. The proposed method does not depend on a prior knowledge of exact rank value. Numerical experiments demonstrate that the proposed homotopy method weakens the affection of the choice of the regularization parameter, and is more efficient and effective than the existing sate-of-the-art methods.

Keywords:

Mathematics Subject Classification: Primary: 15A29, 65F10, 90C27.

Citation:

FullText(HTML)

Table 4. Numerical results on random uniform matrices with size $ m = n = 500 $ and sampling ratio $ \tau = 0.5. $

	r=5				r=10
Alg.	NS	time	rank	rel.err.	NS	time	rank	rel.err.
FPCA	50	6.71	5	9.22E-06	50	13.61	10	2.66E-05
PD	50	24.08	5	7.70E-05	50	33.65	10	8.96E-05
HIHT	50	19.37	5	5.33E-05	50	16.12	10	4.78E-05
	r=30				r=50
Alg.	NS	time	rank	rel.err.	NS	time	rank	rel.err.
FPCA	0	43.74	16.6	3.80E-02	-	-	-	-
PD	50	39.55	30	7.42E-05	50	53.83	50	1.00E-04
HIHT	50	36.34	30	3.52E-05	50	47.36	50	9.93E-06
	r=70				r=90
Alg.	NS	time	rank	rel.err.	NS	time	rank	rel.err.
FPCA	-	-	-	-	-	-	-	-
PD	50	87.17	70	1.51E-04	50	294.05	90	2.38E-04
HIHT	50	70.3	70	6.31E-05	50	82.79	90	2.02E-04

| Show Table

DownLoad: CSV

Table 1. Influence of the regularization parameter $ \lambda $

$ \eta $	IHT				HIHT
$ \eta $	NS	time	rank	rel.err.	NS	time	rank	rel.err.
50	4	122.43	38.56	1.29E-01	50	35.33	40	1.17E-05
20	50	18.83	40.00	5.82E-05	50	19.86	40	1.17E-05
10	47	23.69	40.06	2.52E-03	50	23.25	40	1.38E-05
5	0	52.92	227.10	6.33E-01	0	45.41	227.72	6.34E-01

| Show Table

DownLoad: CSV

Table 2. Influence of maximum number of inner iterations on Algorithm 2

	HIHT
$ step $	NS	time	rank	rel.err.
5	50	16.72	40	4.35E-05
10	50	18.13	40	2.09E-05
20	50	18.97	40	1.35E-05
30	50	19.29	40	1.16E-05
40	50	19.27	40	1.16E-05
50	50	19.24	40	1.17E-05
60	50	19.72	40	1.01E-05
70	50	19.71	40	1.02E-05

| Show Table

DownLoad: CSV

Table 3. Numerical results on random Gaussian matrices with size $ m = n = 500 $ and sampling ratio $ \tau = 0.5 $

	r=5				r=10
Alg.	NS	time	rank	rel.err.	NS	time	rank	rel.err.
FPCA	50	4.69	5	1.24E-06	50	5.19	10	2.90E-06
PD	50	95.92	5	5.92E-05	50	102.83	10	8.54E-05
HIHT	50	10.91	5	7.49E-05	50	12.54	10	7.02E-05
	r=30				r=50
Alg.	NS	time	rank	rel.err.	NS	time	rank	rel.err.
FPCA	50	5.78	30	1.44E-05	50	255.32	50	1.61E-04
PD	50	102.38	30	9.40E-05	50	106.19	50	1.50E-04
HIHT	50	18.57	30	6.63E-05	50	24.13	50	2.67E-05
	r=70				r=90
Alg.	NS	time	rank	rel.err.	NS	time	rank	rel.err.
FPCA	50	279.55	70	2.23E-04	50	332.90	90	2.71E-04
PD	50	164.06	70	1.78E-04	50	471.48	90	2.51E-04
HIHT	50	38.23	70	5.01E-05	50	84.32	90	2.08E-04

| Show Table

DownLoad: CSV

Table 5. Computational results on random Gaussian (or uniform) matrices with size $ m = n = 4000 $, $ r = 200 $, and sampling ratio $ \tau = 0.3. $

	Gaussian matrix				uniform matrix
Alg.	NS	time	rank	rel.err.	NS	time	rank	rel.err.
FPCA	50	284.36	200	8.28E-05	0	1887.44	1.06	2.35E-02
PD	50	13232.85	200	1.02E-04	50	5976.50	200	9.89E-05
HIHT	50	4427.63	200	1.63E-05	50	7637.74	200	1.70E-05

| Show Table

DownLoad: CSV

Related Papers

Cited by

References

[1]	J. D. Blanchard, J. Tanner and K. Wei, CGIHT: Conjugate gradient iterative hard thresholding for compressed sensing and matrix completion, Information and Inference: A Journal of the IMA, 4 (2014), 289-327. doi: 10.1093/imaiai/iav011.
[2]	T. Blumensath and M. E. Davies, Iterative thresholding for sparse approximations, Journal of Constructive Approximation, 14 (2008), 629-654. doi: 10.1007/s00041-008-9035-z.
[3]	T. Blumensath and M. E. Davies, Normalised itertive hard thresholding: guaranteed stability and performance, IEEE Journal of Selected Topics in Signal Processing, 4 (2010), 298-309.
[4]	E. J. Candès and B. Recht, Exact matrix completion via convex optimization, Foundations of Computational Mathematics, 9 (2009), 717-772. doi: 10.1007/s10208-009-9045-5.
[5]	E. J. Candès and T. Tao, The power of convex relaxation: Near-optimal matrix completion, IEEE Transactions on Information Theory, 56 (2009), 2053-1080. doi: 10.1109/TIT.2010.2044061.
[6]	M. Fazel, H. Hindi and S. Boyd, Rank minimization and applications in system theory, Proceedings of the American Control Conference, 4 (2004), 3273-3278.
[7]	M. Fazel, T. K. Pong, D. Sun and P. Tseng, Hankel matrix rank minimization with applications to system identification and realization, SIAM Journal on Matrix Analysis and Applications, 34 (2013), 946-977. doi: 10.1137/110853996.
[8]	D. Goldfarb and S. Ma, Convergence of fixed-point continuation algorithms for matrix rank minimization, Foundations of Computational Mathematics, 11 (2011), 183-210. doi: 10.1007/s10208-011-9084-6.
[9]	J. P. Haldar and D. Hernando, Rank-constrained solutions to linear matrix equations using powerfactorization, IEEE Signal Processing Letters, 16 (2009), 584-587.
[10]	N. J. A. Harvey, D. R. Karger and S. Yekhanin, The complexity of matrix completion, Proceedings of the 17th Annual ACM-SIAM Symposium on Discrete Algorithm, (2006), 1103–1111. doi: 10.1145/1109557.1109679.
[11]	A. Kyrillidis and V. Cevher, Martix ALPS: Accelerated low rank and sparse matrix reconstruction, Technical Report, 2012.
[12]	A. Kyrillidis and V. Cevher, Martix recips for hard thresholding methods, Journal of Mathematical Imaging and Vision, 48 (2014), 235-265. doi: 10.1007/s10851-013-0434-7.
[13]	Y. Liu, J. Tao, H. Zhang, X. Xiu and L. Kong, Fused LASSO penalized least absolute deviation estimator for high dimensional linear regression, Numerical Algebra, Control & Optimization, 8 (2018), 97–117. doi: 10.3934/naco.2018006.
[14]	Z. Liu and L. Vandenberghe, Interior-point method for nuclear norm approximation with application to system identification, SIAM Journal on Matrix Analysis and Applications, 31 (2009), 1235-1256. doi: 10.1137/090755436.
[15]	C. Lu, J. Tang, S. Yan and Z. Lin, Generalized nonconvex nonsmooth low-rank minimization, IEEE Conference on Computer Vision and Pattern Recognition, 2014.
[16]	Z. Lu, Iterative hard thresholding methods for $l_0$ regularized convex cone programming, Mathematical Programming, 147 (2014), 125-154. doi: 10.1007/s10107-013-0714-4.
[17]	Z. Lu and Y. Zhang, Penalty decomposition methods for rank minimization, Optimization Methods and Software, 30 (2015), 531-558. doi: 10.1080/10556788.2014.936438.
[18]	S. Ma, D. Goldfarb and L. Chen, Fixed point and bregman iterative methods for matrix rank minimization, Mathematical Programming, 128 (2011), 321-353. doi: 10.1007/s10107-009-0306-5.
[19]	K. Mohan and M. Fazel, Reweighted nuclear norm minimization with application to system identification, Proceedings of the American Control Conference, 2010.
[20]	K. Mohan and M. Fazel, Iterative reweighted algorithms for matrix rank minimization, Journal of Machine Learning Research, 13 (2012), 3441-3473.
[21]	B. Recht, M. Fazel and P. A. Parrilo, Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization, SIAM Review, 52 (2010), 471-501. doi: 10.1137/070697835.
[22]	J. Tanner and K. Wei, Normalized iterative hard thresholding for matrix completion, SIAM Journal on Scientific Computing, 35 (2013), S104–S125. doi: 10.1137/120876459.
[23]	Z. Wen, W. Yin and Y. Zhang, Solving a low-rank factorization model for matrix completion by a non-linear successive over-relaxation algorithm, Mathematical Programming Computation, 4 (2012), 333-361. doi: 10.1007/s12532-012-0044-1.
[24]	Z. Weng and X. Wang, Low-rank matrix completion for array signal processing, IEEE International Conference on Speech and Signal Processing, (2012), 2697–2700.