American Institute of Mathematical Sciences

Advanced
May  2015, 9(2): 601-624. doi: 10.3934/ipi.2015.9.601

Parallel matrix factorization for low-rank tensor completion

Yangyang Xu 1, , Ruru Hao 2, , Wotao Yin 3, and  Zhixun Su 2,

1. 

Department of Computational and Applied Mathematics, Rice University, Houston, TX 77005
2. 

School of Mathematical Sciences, Dalian University of Technology, Dalian, China, China
3. 

Department of Mathematics, University of California, Los Angeles, CA 90095-1555

Received  December 2013 Revised  October 2014 Published  March 2015

Higher-order low-rank tensors naturally arise in many applications including hyperspectral data recovery, video inpainting, seismic data reconstruction, and so on. We propose a new model to recover a low-rank tensor by simultaneously performing low-rank matrix factorizations to the all-mode matricizations of the underlying tensor. An alternating minimization algorithm is applied to solve the model, along with two adaptive rank-adjusting strategies when the exact rank is not known.
    Phase transition plots reveal that our algorithm can recover a variety of synthetic low-rank tensors from significantly fewer samples than the compared methods, which include a matrix completion method applied to tensor recovery and two state-of-the-art tensor completion methods. Further tests on real-world data show similar advantages. Although our model is non-convex, our algorithm performs consistently throughout the tests and gives better results than the compared methods, some of which are based on convex models. In addition, subsequence convergence of our algorithm can be established in the sense that any limit point of the iterates satisfies the KKT condtions.
Keywords: low-rank tensor completion, low-rank matrix completion, alternating least squares, non-convex optimization., Higher-order tensor.
Mathematics Subject Classification: Primary: 94A08, 94A12; Secondary: 90C9.
Citation: Yangyang Xu, Ruru Hao, Wotao Yin, Zhixun Su. Parallel matrix factorization for low-rank tensor completion. Inverse Problems and Imaging, 2015, 9 (2) : 601-624. doi: 10.3934/ipi.2015.9.601
References:
[1]

J. Cai, E. Candes and Z. Shen, A singular value thresholding algorithm for matrix completion, SIAM J. Optim., 20 (2010), 1956-1982. doi: 10.1137/080738970.

[2]

E. Candès and B. Recht, Exact matrix completion via convex optimization, Foundations of Computational Mathematics, 9 (2009), 717-772. doi: 10.1007/s10208-009-9045-5.

[3]

C. Chen, B. He and X. Yuan, Matrix completion via an alternating direction method, IMA Journal of Numerical Analysis, 32 (2012), 227-245. doi: 10.1093/imanum/drq039.

[4]

S. Gandy, B. Recht and I. Yamada, Tensor completion and low-n-rank tensor recovery via convex optimization, Inverse Problems, 27 (2011), 025010, 1-19. doi: 10.1088/0266-5611/27/2/025010.

[5]

L. Grippo and M. Sciandrone, On the convergence of the block nonlinear Gauss-Seidel method under convex constraints, Oper. Res. Lett., 26 (2000), 127-136. doi: 10.1016/S0167-6377(99)00074-7.

[6]

B. Jiang, S. Ma and S. Zhang, Tensor principal component analysis via convex optimization, Mathematical Programming, (2014), 1-35. doi: 10.1007/s10107-014-0774-0.

[7]

H. A. L. Kiers, Towards a standardized notation and terminology in multiway analysis, Journal of Chemometrics, 14 (2000), 105-122. doi: 10.1002/1099-128X(200005/06)14:3<105::AID-CEM582>3.0.CO;2-I.

[8]

M. Kilmer, K. Braman, N. Hao and R. Hoover, Third-order tensors as operators on matrices: A theoretical and computational framework with applications in imaging, SIAM Journal on Matrix Analysis and Applications, 34 (2013), 148-172. doi: 10.1137/110837711.

[9]

T. G. Kolda and B. W. Bader, Tensor decompositions and applications, SIAM review, 51 (2009), 455-500. doi: 10.1137/07070111X.

[10]

T. G. Kolda, B. W. Bader and J. P. Kenny, Higher-order web link analysis using multilinear algebra, in Data Mining, Fifth IEEE International Conference on, IEEE, 2005. doi: 10.1109/ICDM.2005.77.

[11]

N. Kreimer and M. D. Sacchi, A tensor higher-order singular value decomposition for prestack seismic data noise reduction and interpolation, Geophysics, 77 (2012), V113-V122. doi: 10.1190/geo2011-0399.1.

[12]

D. Kressner, M. Steinlechner and B. Vandereycken, Low-rank tensor completion by riemannian optimization, BIT, 54 (2014), 447-468. doi: 10.1007/s10543-013-0455-z.

[13]

M. Lai, Y. Xu and W. Yin, Improved iteratively reweighted least squares for unconstrained smoothed $l_q$ minimization, SIAM Journal on Numerical Analysis, 51 (2013), 927-957. doi: 10.1137/110840364.

[14]

N. Li and B. Li, Tensor completion for on-board compression of hyperspectral images, in 2010 17th IEEE International Conference on Image Processing (ICIP), IEEE, 2010, 517-520. doi: 10.1109/ICIP.2010.5651225.

[15]

Q. Ling, Y. Xu, W. Yin and Z. Wen, Decentralized low-rank matrix completion, in 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2012, 2925-2928. doi: 10.1109/ICASSP.2012.6288528.

[16]

J. Liu, P. Musialski, P. Wonka and J. Ye, Tensor completion for estimating missing values in visual data, IEEE Transactions on Pattern Analysis and Machine Intelligence, (2012), 208-220. doi: 10.1109/TPAMI.2012.39.

[17]

S. Ma, D. Goldfarb and L. Chen, Fixed point and bregman iterative methods for matrix rank minimization, Mathematical Programming, 128 (2011), 321-353. doi: 10.1007/s10107-009-0306-5.

[18]

C. Mu, B. Huang, J. Wright and D. Goldfarb, Square deal: Lower bounds and improved relaxations for tensor recovery, preprint, arXiv:1307.5870, (2013).

[19]

K. A. Patwardhan, G. Sapiro and M. Bertalmío, Video inpainting under constrained camera motion, IEEE Transactions on Image Processing, 16 (2007), 545-553. doi: 10.1109/TIP.2006.888343.

[20]

B. Recht, M. Fazel and P. A. Parrilo, Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization, SIAM Review, 52 (2010), 471-501. doi: 10.1137/070697835.

[21]

B. Romera-Paredes and M. Pontil, A new convex relaxation for tensor completion, preprint, arXiv:1307.4653, (2013).

[22]

A. C. Sauve, A. O. Hero III, W. Leslie Rogers, S. J. Wilderman and N. H. Clinthorne, 3d image reconstruction for a compton spect camera model, Nuclear Science, IEEE Transactions on, 46 (1999), 2075-2084. doi: 10.1109/23.819285.

[23]

J. Sun, H. Zeng, H. Liu, Y. Lu and Z. Chen, Cubesvd: a novel approach to personalized web search, in Proceedings of the 14th international conference on World Wide Web, ACM, 2005, 382-390. doi: 10.1145/1060745.1060803.

[24]

K. C. Toh and S. Yun, An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems, Pacific Journal of Optimization, 6 (2010), 615-640.

[25]

P. Tseng, Convergence of a block coordinate descent method for nondifferentiable minimization, Journal of Optimization Theory and Applications, 109 (2001), 475-494. doi: 10.1023/A:1017501703105.

[26]

L. R. Tucker, Some mathematical notes on three-mode factor analysis, Psychometrika, 31 (1966), 279-311. doi: 10.1007/BF02289464.

[27]

Z. Wen, W. Yin and Y. Zhang, Solving a low-rank factorization model for matrix completion by a nonlinear successive over-relaxation algorithm, Mathematical Programming Computation, 4 (2012), 333-361. doi: 10.1007/s12532-012-0044-1.

[28]

Z. Xing, M. Zhou, A. Castrodad, G. Sapiro and L. Carin, Dictionary learning for noisy and incomplete hyperspectral images, SIAM Journal on Imaging Sciences, 5 (2012), 33-56. doi: 10.1137/110837486.

[29]

Y. Xu and W. Yin, A block coordinate descent method for regularized multi-convex optimization with applications to nonnegative tensor factorization and completion, SIAM Journal on Imaging Sciences, 6 (2013), 1758-1789. doi: 10.1137/120887795.

[30]

Y. Xu and W. Yin, A globally convergent algorithm for nonconvex optimization based on block coordinate update, arXiv:1410.1386, (2014).

[31]

Y. Xu, W. Yin, Z. Wen and Y. Zhang, An alternating direction algorithm for matrix completion with nonnegative factors, Journal of Frontiers of Mathematics in China, Special Issue on Computational Mathematics, 7 (2011), 365-384. doi: 10.1007/s11464-012-0194-5.

[32]

Z. Zhang, G. Ely, S. Aeron, N. Hao and M. Kilmer, Novel factorization strategies for higher order tensors: Implications for compression and recovery of multi-linear data, preprint, arXiv:1307.0805v3, (2013).

show all references

References:
[1]

J. Cai, E. Candes and Z. Shen, A singular value thresholding algorithm for matrix completion, SIAM J. Optim., 20 (2010), 1956-1982. doi: 10.1137/080738970.

[2]

E. Candès and B. Recht, Exact matrix completion via convex optimization, Foundations of Computational Mathematics, 9 (2009), 717-772. doi: 10.1007/s10208-009-9045-5.

[3]

C. Chen, B. He and X. Yuan, Matrix completion via an alternating direction method, IMA Journal of Numerical Analysis, 32 (2012), 227-245. doi: 10.1093/imanum/drq039.

[4]

S. Gandy, B. Recht and I. Yamada, Tensor completion and low-n-rank tensor recovery via convex optimization, Inverse Problems, 27 (2011), 025010, 1-19. doi: 10.1088/0266-5611/27/2/025010.

[5]

L. Grippo and M. Sciandrone, On the convergence of the block nonlinear Gauss-Seidel method under convex constraints, Oper. Res. Lett., 26 (2000), 127-136. doi: 10.1016/S0167-6377(99)00074-7.

[6]

B. Jiang, S. Ma and S. Zhang, Tensor principal component analysis via convex optimization, Mathematical Programming, (2014), 1-35. doi: 10.1007/s10107-014-0774-0.

[7]

H. A. L. Kiers, Towards a standardized notation and terminology in multiway analysis, Journal of Chemometrics, 14 (2000), 105-122. doi: 10.1002/1099-128X(200005/06)14:3<105::AID-CEM582>3.0.CO;2-I.

[8]

M. Kilmer, K. Braman, N. Hao and R. Hoover, Third-order tensors as operators on matrices: A theoretical and computational framework with applications in imaging, SIAM Journal on Matrix Analysis and Applications, 34 (2013), 148-172. doi: 10.1137/110837711.

[9]

T. G. Kolda and B. W. Bader, Tensor decompositions and applications, SIAM review, 51 (2009), 455-500. doi: 10.1137/07070111X.

[10]

T. G. Kolda, B. W. Bader and J. P. Kenny, Higher-order web link analysis using multilinear algebra, in Data Mining, Fifth IEEE International Conference on, IEEE, 2005. doi: 10.1109/ICDM.2005.77.

[11]

N. Kreimer and M. D. Sacchi, A tensor higher-order singular value decomposition for prestack seismic data noise reduction and interpolation, Geophysics, 77 (2012), V113-V122. doi: 10.1190/geo2011-0399.1.

[12]

D. Kressner, M. Steinlechner and B. Vandereycken, Low-rank tensor completion by riemannian optimization, BIT, 54 (2014), 447-468. doi: 10.1007/s10543-013-0455-z.

[13]

M. Lai, Y. Xu and W. Yin, Improved iteratively reweighted least squares for unconstrained smoothed $l_q$ minimization, SIAM Journal on Numerical Analysis, 51 (2013), 927-957. doi: 10.1137/110840364.

[14]

N. Li and B. Li, Tensor completion for on-board compression of hyperspectral images, in 2010 17th IEEE International Conference on Image Processing (ICIP), IEEE, 2010, 517-520. doi: 10.1109/ICIP.2010.5651225.

[15]

Q. Ling, Y. Xu, W. Yin and Z. Wen, Decentralized low-rank matrix completion, in 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2012, 2925-2928. doi: 10.1109/ICASSP.2012.6288528.

[16]

J. Liu, P. Musialski, P. Wonka and J. Ye, Tensor completion for estimating missing values in visual data, IEEE Transactions on Pattern Analysis and Machine Intelligence, (2012), 208-220. doi: 10.1109/TPAMI.2012.39.

[17]

S. Ma, D. Goldfarb and L. Chen, Fixed point and bregman iterative methods for matrix rank minimization, Mathematical Programming, 128 (2011), 321-353. doi: 10.1007/s10107-009-0306-5.

[18]

C. Mu, B. Huang, J. Wright and D. Goldfarb, Square deal: Lower bounds and improved relaxations for tensor recovery, preprint, arXiv:1307.5870, (2013).

[19]

K. A. Patwardhan, G. Sapiro and M. Bertalmío, Video inpainting under constrained camera motion, IEEE Transactions on Image Processing, 16 (2007), 545-553. doi: 10.1109/TIP.2006.888343.

[20]

B. Recht, M. Fazel and P. A. Parrilo, Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization, SIAM Review, 52 (2010), 471-501. doi: 10.1137/070697835.

[21]

B. Romera-Paredes and M. Pontil, A new convex relaxation for tensor completion, preprint, arXiv:1307.4653, (2013).

[22]

A. C. Sauve, A. O. Hero III, W. Leslie Rogers, S. J. Wilderman and N. H. Clinthorne, 3d image reconstruction for a compton spect camera model, Nuclear Science, IEEE Transactions on, 46 (1999), 2075-2084. doi: 10.1109/23.819285.

[23]

J. Sun, H. Zeng, H. Liu, Y. Lu and Z. Chen, Cubesvd: a novel approach to personalized web search, in Proceedings of the 14th international conference on World Wide Web, ACM, 2005, 382-390. doi: 10.1145/1060745.1060803.

[24]

K. C. Toh and S. Yun, An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems, Pacific Journal of Optimization, 6 (2010), 615-640.

[25]

P. Tseng, Convergence of a block coordinate descent method for nondifferentiable minimization, Journal of Optimization Theory and Applications, 109 (2001), 475-494. doi: 10.1023/A:1017501703105.

[26]

L. R. Tucker, Some mathematical notes on three-mode factor analysis, Psychometrika, 31 (1966), 279-311. doi: 10.1007/BF02289464.

[27]

Z. Wen, W. Yin and Y. Zhang, Solving a low-rank factorization model for matrix completion by a nonlinear successive over-relaxation algorithm, Mathematical Programming Computation, 4 (2012), 333-361. doi: 10.1007/s12532-012-0044-1.

[28]

Z. Xing, M. Zhou, A. Castrodad, G. Sapiro and L. Carin, Dictionary learning for noisy and incomplete hyperspectral images, SIAM Journal on Imaging Sciences, 5 (2012), 33-56. doi: 10.1137/110837486.

[29]

Y. Xu and W. Yin, A block coordinate descent method for regularized multi-convex optimization with applications to nonnegative tensor factorization and completion, SIAM Journal on Imaging Sciences, 6 (2013), 1758-1789. doi: 10.1137/120887795.

[30]

Y. Xu and W. Yin, A globally convergent algorithm for nonconvex optimization based on block coordinate update, arXiv:1410.1386, (2014).

[31]

Y. Xu, W. Yin, Z. Wen and Y. Zhang, An alternating direction algorithm for matrix completion with nonnegative factors, Journal of Frontiers of Mathematics in China, Special Issue on Computational Mathematics, 7 (2011), 365-384. doi: 10.1007/s11464-012-0194-5.

[32]

Z. Zhang, G. Ely, S. Aeron, N. Hao and M. Kilmer, Novel factorization strategies for higher order tensors: Implications for compression and recovery of multi-linear data, preprint, arXiv:1307.0805v3, (2013).

[1]

Kaixin Gao, Zheng-Hai Huang, Xiaolei Liu. Enhancing low-rank tensor completion via first-order and second-order total variation regularizations. Journal of Industrial and Management Optimization, 2022  doi: 10.3934/jimo.2022136
[2]

Dan Zhu, Rosemary A. Renaut, Hongwei Li, Tianyou Liu. Fast non-convex low-rank matrix decomposition for separation of potential field data using minimal memory. Inverse Problems and Imaging, 2021, 15 (1) : 159-183. doi: 10.3934/ipi.2020076
[3]

Ke Wei, Jian-Feng Cai, Tony F. Chan, Shingyu Leung. Guarantees of riemannian optimization for low rank matrix completion. Inverse Problems and Imaging, 2020, 14 (2) : 233-265. doi: 10.3934/ipi.2020011
[4]

Yun Cai, Song Li. Convergence and stability of iteratively reweighted least squares for low-rank matrix recovery. Inverse Problems and Imaging, 2017, 11 (4) : 643-661. doi: 10.3934/ipi.2017030
[5]

Wei Wan, Weihong Guo, Jun Liu, Haiyang Huang. Non-local blind hyperspectral image super-resolution via 4d sparse tensor factorization and low-rank. Inverse Problems and Imaging, 2020, 14 (2) : 339-361. doi: 10.3934/ipi.2020015
[6]

Tao Wu, Yu Lei, Jiao Shi, Maoguo Gong. An evolutionary multiobjective method for low-rank and sparse matrix decomposition. Big Data & Information Analytics, 2017, 2 (1) : 23-37. doi: 10.3934/bdia.2017006
[7]

Zhouchen Lin. A review on low-rank models in data analysis. Big Data & Information Analytics, 2016, 1 (2&3) : 139-161. doi: 10.3934/bdia.2016001
[8]

Meng Ding, Ting-Zhu Huang, Xi-Le Zhao, Michael K. Ng, Tian-Hui Ma. Tensor train rank minimization with nonlocal self-similarity for tensor completion. Inverse Problems and Imaging, 2021, 15 (3) : 475-498. doi: 10.3934/ipi.2021001
[9]

Simon Foucart, Richard G. Lynch. Recovering low-rank matrices from binary measurements. Inverse Problems and Imaging, 2019, 13 (4) : 703-720. doi: 10.3934/ipi.2019032
[10]

Simon Arridge, Pascal Fernsel, Andreas Hauptmann. Joint reconstruction and low-rank decomposition for dynamic inverse problems. Inverse Problems and Imaging, 2022, 16 (3) : 483-523. doi: 10.3934/ipi.2021059
[11]

Kevin N. Webster. Low-rank kernel approximation of Lyapunov functions using neural networks. Journal of Computational Dynamics, 2022  doi: 10.3934/jcd.2022026
[12]

Qilin Wang, Liu He, Shengjie Li. Higher-order weak radial epiderivatives and non-convex set-valued optimization problems. Journal of Industrial and Management Optimization, 2019, 15 (2) : 465-480. doi: 10.3934/jimo.2018051
[13]

Yuan Shen, Xin Liu. An alternating minimization method for matrix completion problems. Discrete and Continuous Dynamical Systems - S, 2020, 13 (6) : 1757-1772. doi: 10.3934/dcdss.2020103
[14]

Qun Liu, Lihua Fu, Meng Zhang, Wanjuan Zhang. Two-dimensional seismic data reconstruction using patch tensor completion. Inverse Problems and Imaging, 2020, 14 (6) : 985-1000. doi: 10.3934/ipi.2020052
[15]

Huiyuan Guo, Quan Yu, Xinzhen Zhang, Lulu Cheng. Low rank matrix minimization with a truncated difference of nuclear norm and Frobenius norm regularization. Journal of Industrial and Management Optimization, 2022  doi: 10.3934/jimo.2022045
[16]

Mingchao Zhao, You-Wei Wen, Michael Ng, Hongwei Li. A nonlocal low rank model for poisson noise removal. Inverse Problems and Imaging, 2021, 15 (3) : 519-537. doi: 10.3934/ipi.2021003
[17]

Hui Zhang, Jian-Feng Cai, Lizhi Cheng, Jubo Zhu. Strongly convex programming for exact matrix completion and robust principal component analysis. Inverse Problems and Imaging, 2012, 6 (2) : 357-372. doi: 10.3934/ipi.2012.6.357
[18]

Henry Adams, Lara Kassab, Deanna Needell. An adaptation for iterative structured matrix completion. Foundations of Data Science, 2021, 3 (4) : 769-791. doi: 10.3934/fods.2021028
[19]

Yong Wang, Wanquan Liu, Guanglu Zhou. An efficient algorithm for non-convex sparse optimization. Journal of Industrial and Management Optimization, 2019, 15 (4) : 2009-2021. doi: 10.3934/jimo.2018134
[20]

H. D. Scolnik, N. E. Echebest, M. T. Guardarucci. Extensions of incomplete oblique projections method for solving rank-deficient least-squares problems. Journal of Industrial and Management Optimization, 2009, 5 (2) : 175-191. doi: 10.3934/jimo.2009.5.175

2021 Impact Factor: 1.483

Tools

Metrics

  • PDF downloads (406)
  • HTML views (0)
  • Cited by (84)

Other articles
by authors

[Back to Top]

Copyright © 2022 American Institute of Mathematical Sciences | Privacy Policy