# American Institute of Mathematical Sciences

April  2020, 14(2): 339-361. doi: 10.3934/ipi.2020015

## Non-local blind hyperspectral image super-resolution via 4d sparse tensor factorization and low-rank

 1 Laboratory of Mathematics and Complex Systems (Ministry of Education of China), School of Mathematical Sciences, Beijing Normal University, Beijing 100875, China 2 Department of Mathematics, Case Western Reserve University, Cleveland, OH, 44106, USA

* Corresponding author: Weihong Guo

Received  June 2019 Revised  October 2019 Published  February 2020

Hyperspectral image (HSI) super-resolution is a technique to improve the spatial resolution of a HSI for better visual perception and down stream applications. This is a very ill-posed inverse problem and is often solved by fusing the low-resolution (LR) HSI with a high-resolution (HR) multispectral image (MSI). It is more challenging for blind HSI super-resolution, i.e., when the spatial degradation operators are completely unknown. In this paper, we propose a novel sparse tensor factorization model for the task of blind HSI super-resolution using the spatial non-local self-similarity and spectral global correlation of HSIs. Image clustering method is employed to collect some similar 3D cubes of HSIs which can be formed as some 4D image clusters with high correlation. We conduct cluster wise computation to not only save computation time but also to introduce a non-local regularity originated from the redundancy of cubes. By using the sparsity of tensor decomposition and the low-rank in non-local self-similarity direction underlying 4D similar clusters, we design a sparse tensor regularization term, which preserves the spatial-spectral structural correlation of HSIs. In addition, we present a proximal alternating direction method of multipliers (ADMM) based algorithm to efficiently solve the proposed model. Numerical experiments demonstrate that the proposed model outperforms many state-of-the-art HSI super-resolution methods.

Citation: Wei Wan, Weihong Guo, Jun Liu, Haiyang Huang. Non-local blind hyperspectral image super-resolution via 4d sparse tensor factorization and low-rank. Inverse Problems & Imaging, 2020, 14 (2) : 339-361. doi: 10.3934/ipi.2020015
##### References:

show all references

##### References:
Illustration of the hyperspectral image super-resolution task
The SAMs of all competing methods on CAVE dataset with uniform blur
The reconstructed images and the corresponding error images of fake and real peppers at the $8^{th}$ band and flowers at the $14^{th}$ band with Gaussian blur and scaling factor 8
The reconstructed images and the corresponding error images of the Indian Pines image at the $60^{th}$ and $130^{th}$ bands with uniform blur and scaling factor 8
The functional energy curve for the proposed algorithm
The reconstructed images and the corresponding error images of the Pavia image at the $45^{th}$ and $75^{th}$ bands with uniform blur and scaling factor 8
Comparison on the average values of each of the five quantitative measures on 32 scenes from CAVE dataset with scaling factor 8
 Methods PSNR SSIM RMSE SAM ERGAS Uniform blur CNMF[37] 43.052 0.985 2.299 5.934 1.224 Hysure[24] 41.461 0.975 2.702 10.737 1.497 STEREO[14] 39.680 0.946 3.611 12.212 1.898 Proposed 45.332 0.987 1.822 4.969 0.984 Gaussian blur CNMF[37] 42.258 0.981 2.511 6.578 1.329 Hysure[24] 41.961 0.977 2.541 10.310 1.425 STEREO[14] 39.542 0.945 3.658 12.251 1.923 Proposed 45.206 0.987 1.847 4.983 0.995
 Methods PSNR SSIM RMSE SAM ERGAS Uniform blur CNMF[37] 43.052 0.985 2.299 5.934 1.224 Hysure[24] 41.461 0.975 2.702 10.737 1.497 STEREO[14] 39.680 0.946 3.611 12.212 1.898 Proposed 45.332 0.987 1.822 4.969 0.984 Gaussian blur CNMF[37] 42.258 0.981 2.511 6.578 1.329 Hysure[24] 41.961 0.977 2.541 10.310 1.425 STEREO[14] 39.542 0.945 3.658 12.251 1.923 Proposed 45.206 0.987 1.847 4.983 0.995
The performance comparison of the methods on the Indian Pines image with scaling factor 8
 Methods PSNR SSIM RMSE SAM ERGAS Uniform blur CNMF[37] 42.866 0.978 2.557 2.394 9.406 Hysure[24] 45.232 0.984 1.891 1.701 9.389 STEREO[14] 48.212 0.985 1.267 1.225 9.405 Proposed 51.441 0.993 0.928 0.888 9.368 Gaussian blur CNMF[37] 42.740 0.977 2.599 2.430 9.409 Hysure[24] 45.927 0.986 1.703 1.547 9.384 STEREO[14] 47.050 0.981 1.510 1.413 9.438 Proposed 51.326 0.993 0.950 0.904 9.369
 Methods PSNR SSIM RMSE SAM ERGAS Uniform blur CNMF[37] 42.866 0.978 2.557 2.394 9.406 Hysure[24] 45.232 0.984 1.891 1.701 9.389 STEREO[14] 48.212 0.985 1.267 1.225 9.405 Proposed 51.441 0.993 0.928 0.888 9.368 Gaussian blur CNMF[37] 42.740 0.977 2.599 2.430 9.409 Hysure[24] 45.927 0.986 1.703 1.547 9.384 STEREO[14] 47.050 0.981 1.510 1.413 9.438 Proposed 51.326 0.993 0.950 0.904 9.369
The performance comparison of noisy cases on the Pavia image with uniform blur and scaling factor 8
 Methods PSNR SSIM RMSE SAM ERGAS SNRh=35dB, SNRm=40dB CNMF[37] 37.958 0.982 3.335 2.790 0.958 Hysure[24] 42.573 0.987 2.278 2.420 0.584 STEREO[14] 37.307 0.951 3.715 4.022 1.039 Proposed 43.731 0.989 1.848 1.944 0.497 SNRh=30dB, SNRm=35dB CNMF[37] 37.694 0.979 3.439 2.885 0.979 Hysure[24] 41.541 0.982 2.338 2.553 0.644 STEREO[14] 35.584 0.932 4.401 4.829 1.252 Proposed 42.097 0.981 2.158 2.349 0.604
 Methods PSNR SSIM RMSE SAM ERGAS SNRh=35dB, SNRm=40dB CNMF[37] 37.958 0.982 3.335 2.790 0.958 Hysure[24] 42.573 0.987 2.278 2.420 0.584 STEREO[14] 37.307 0.951 3.715 4.022 1.039 Proposed 43.731 0.989 1.848 1.944 0.497 SNRh=30dB, SNRm=35dB CNMF[37] 37.694 0.979 3.439 2.885 0.979 Hysure[24] 41.541 0.982 2.338 2.553 0.644 STEREO[14] 35.584 0.932 4.401 4.829 1.252 Proposed 42.097 0.981 2.158 2.349 0.604
 [1] Amine Laghrib, Abdelkrim Chakib, Aissam Hadri, Abdelilah Hakim. A nonlinear fourth-order PDE for multi-frame image super-resolution enhancement. Discrete & Continuous Dynamical Systems - B, 2020, 25 (1) : 415-442. doi: 10.3934/dcdsb.2019188 [2] Yangyang Xu, Ruru Hao, Wotao Yin, Zhixun Su. Parallel matrix factorization for low-rank tensor completion. Inverse Problems & Imaging, 2015, 9 (2) : 601-624. doi: 10.3934/ipi.2015.9.601 [3] Michael K. Ng, Chi-Pan Tam, Fan Wang. Multi-view foreground segmentation via fourth order tensor learning. Inverse Problems & Imaging, 2013, 7 (3) : 885-906. doi: 10.3934/ipi.2013.7.885 [4] Jan Boman, Vladimir Sharafutdinov. Stability estimates in tensor tomography. Inverse Problems & Imaging, 2018, 12 (5) : 1245-1262. doi: 10.3934/ipi.2018052 [5] Houssem Haddar, Alexander Konschin. Factorization method for imaging a local perturbation in inhomogeneous periodic layers from far field measurements. Inverse Problems & Imaging, 2020, 14 (1) : 133-152. doi: 10.3934/ipi.2019067 [6] Qiyu Jin, Ion Grama, Quansheng Liu. Convergence theorems for the Non-Local Means filter. Inverse Problems & Imaging, 2018, 12 (4) : 853-881. doi: 10.3934/ipi.2018036 [7] Gabriel Peyré, Sébastien Bougleux, Laurent Cohen. Non-local regularization of inverse problems. Inverse Problems & Imaging, 2011, 5 (2) : 511-530. doi: 10.3934/ipi.2011.5.511 [8] Olivier Bonnefon, Jérôme Coville, Guillaume Legendre. Concentration phenomenon in some non-local equation. Discrete & Continuous Dynamical Systems - B, 2017, 22 (3) : 763-781. doi: 10.3934/dcdsb.2017037 [9] Sirui Li, Wei Wang, Pingwen Zhang. Local well-posedness and small Deborah limit of a molecule-based $Q$-tensor system. Discrete & Continuous Dynamical Systems - B, 2015, 20 (8) : 2611-2655. doi: 10.3934/dcdsb.2015.20.2611 [10] Mengmeng Zheng, Ying Zhang, Zheng-Hai Huang. Global error bounds for the tensor complementarity problem with a P-tensor. Journal of Industrial & Management Optimization, 2019, 15 (2) : 933-946. doi: 10.3934/jimo.2018078 [11] Yiju Wang, Guanglu Zhou, Louis Caccetta. Nonsingular $H$-tensor and its criteria. Journal of Industrial & Management Optimization, 2016, 12 (4) : 1173-1186. doi: 10.3934/jimo.2016.12.1173 [12] Henri Berestycki, Nancy Rodríguez. A non-local bistable reaction-diffusion equation with a gap. Discrete & Continuous Dynamical Systems - A, 2017, 37 (2) : 685-723. doi: 10.3934/dcds.2017029 [13] Chiu-Yen Kao, Yuan Lou, Wenxian Shen. Random dispersal vs. non-local dispersal. Discrete & Continuous Dynamical Systems - A, 2010, 26 (2) : 551-596. doi: 10.3934/dcds.2010.26.551 [14] Hongjie Dong, Doyoon Kim. Schauder estimates for a class of non-local elliptic equations. Discrete & Continuous Dynamical Systems - A, 2013, 33 (6) : 2319-2347. doi: 10.3934/dcds.2013.33.2319 [15] Matteo Focardi. Vector-valued obstacle problems for non-local energies. Discrete & Continuous Dynamical Systems - B, 2012, 17 (2) : 487-507. doi: 10.3934/dcdsb.2012.17.487 [16] Tao Wang. Global dynamics of a non-local delayed differential equation in the half plane. Communications on Pure & Applied Analysis, 2014, 13 (6) : 2475-2492. doi: 10.3934/cpaa.2014.13.2475 [17] Jared C. Bronski, Razvan C. Fetecau, Thomas N. Gambill. A note on a non-local Kuramoto-Sivashinsky equation. Discrete & Continuous Dynamical Systems - A, 2007, 18 (4) : 701-707. doi: 10.3934/dcds.2007.18.701 [18] Niels Jacob, Feng-Yu Wang. Higher order eigenvalues for non-local Schrödinger operators. Communications on Pure & Applied Analysis, 2018, 17 (1) : 191-208. doi: 10.3934/cpaa.2018012 [19] Rafael Abreu, Cristian Morales-Rodrigo, Antonio Suárez. Some eigenvalue problems with non-local boundary conditions and applications. Communications on Pure & Applied Analysis, 2014, 13 (6) : 2465-2474. doi: 10.3934/cpaa.2014.13.2465 [20] Walter Allegretto, Yanping Lin, Shuqing Ma. On the box method for a non-local parabolic variational inequality. Discrete & Continuous Dynamical Systems - B, 2001, 1 (1) : 71-88. doi: 10.3934/dcdsb.2001.1.71

2019 Impact Factor: 1.373