Image retinex based on the nonconvex TV-type regularization

Yuan Wang; Zhi-Feng Pang; Yuping Duan; Ke Chen

doi:10.3934/ipi.2020050

Article Contents

2021, Volume 15, Issue 6: 1381-1407. Doi: 10.3934/ipi.2020050 open access

This issue Previous Article LANTERN: Learn analysis transform network for dynamic magnetic resonance imaging Next Article Image fusion network for dual-modal restoration

Image retinex based on the nonconvex TV-type regularization

1.
School of Mathematics and Statistics, Henan University, Kaifeng, 475004, China
2.
Center for Applied Mathematics, Tianjin University, Tianjin, 300072, China
3.
Department of Mathematical Sciences, University of Liverpool, L693BX, United Kingdom

^* Corresponding author: Zhi-Feng Pang
^* Corresponding author: Zhi-Feng Pang

Received: September 30, 2019

Revised: April 30, 2020

Early access: August 2020

Published: December 2021

Dr. Z.-F. Pang was partially supported by National Basic Research Program of China (973 Program No.2015CB856003), and also gratefully acknowledges financial support from China Scholarship Council(CSC) as a research scholar to visit the University of Liverpool from August 2017 to August 2018

Abstract Full Text(HTML) Figure(17) / Table(3) Related Papers Cited by

Abstract

Retinex theory is introduced to show how the human visual system perceives the color and the illumination effect such as Retinex illusions, medical image intensity inhomogeneity and color shadow effect etc.. Many researchers have studied this ill-posed problem based on the framework of the variation energy functional for decades. However, to the best of our knowledge, the existing models via the sparsity of the image based on the nonconvex $ \ell^p $-quasinorm were limited. To deal with this problem, this paper considers a TV$ _p $-HOTV$ _q $-based retinex model with $ p, q\in(0, 1) $. Specially, the TV$ _p $ term based on the total variation(TV) regularization can describe the reflectance efficiently, which has the piecewise constant structure. The HOTV$ _q $ term based on the high order total variation(HOTV) regularization can penalize the smooth structure called the illumination. Since the proposed model is non-convex, non-smooth and non-Lipschitz, we employ the iteratively reweighed $ \ell_1 $ (IRL1) algorithm to solve it. We also discuss some properties of our proposed model and algorithm. Experimental experiments on the simulated and real images illustrate the effectiveness and the robustness of our proposed model both visually and quantitatively by compared with some related state-of-the-art variational models.

Keywords:

Image Retinex,
TV$ _p $-HOTV$ _q $ Regularization,
Iteratively Reweighed $ \ell_1 $ Algorithm,
Alternating Minimization Method,
Bias Field Correction.

Mathematics Subject Classification: 80M30, 80M50, 68U10.

Citation:

FullText(HTML)

Figure 1. Two synthetic testing images

Download: Full-size image PowerPoint slide

Figure 2. Reconstruction images based on the different model. The first and third rows: recovered reflectance $ r $ by five methods. The second and fourth rows: illumination $ l $ by five methods

Download: Full-size image PowerPoint slide

Figure 3. Comparison of five methods on $ T_1 $-weighted brain MRIs with different levels of noises

Download: Full-size image PowerPoint slide

Figure 4. Comparison among five models in terms of the PSNR and the MSSIM. Here the x-axis denotes the the serial number of the image

Download: Full-size image PowerPoint slide

Figure 5. Comparison of the performance in terms of CV(%)

Download: Full-size image PowerPoint slide

Figure 6. $ T_1 $-weighted brain images with different intensity inhomogeneities and noises

Download: Full-size image PowerPoint slide

Figure 7. Performances of five methods on $ T_1 $-weighted brain images with different intensity inhomogeneities for the first, third and fifth rows; Colorbar to different between the clean images and restored images for the second, fourth and sixth rows

Download: Full-size image PowerPoint slide

Figure 8. The relative errors of the original images and the restored images in Figure 6.(a) First image, (b) Second image and (c) Third image

Download: Full-size image PowerPoint slide

Figure 9. The relative errors of $ r $ and $ l $ and numerical energy of our model for the third image in Figure 6

Download: Full-size image PowerPoint slide

Figure 10. MRIs with noises and bias field

Download: Full-size image PowerPoint slide

Figure 11. Bias field correction for the different MRIs

Download: Full-size image PowerPoint slide

Figure 12. The gray values of underlined part with five methods

Download: Full-size image PowerPoint slide

Figure 13. Original corrupted images and the corresponding contour images

Download: Full-size image PowerPoint slide

Figure 14. Comparisons of restored images with five methods. The head 1 for the first row, the head 2 for the second row and the head 3 for the third row

Download: Full-size image PowerPoint slide

Figure 15. FIGURE 14 corresponding contour images. The head 1 for the first row, the head 2 for the second row and the head 3 for the third row

Download: Full-size image PowerPoint slide

Figure 16. Two test images for the illusion problem. (a) Adelson's checkerboard shadow image. (b) Logvinenko's cube shadow image

Download: Full-size image PowerPoint slide

Figure 17. Decomposition comparison of the checkerboard image and the cube image. The first and third rows: recovered reflectance $ r $ by five methods. The second and fourth rows: illumination $ l $ by five methods

Download: Full-size image PowerPoint slide

Table 1. PSNR and MSSIM of the reconstructed synthetic images

		PSNR	MSSIM
	TVH1	30.9323	0.9812
Shape1	L0MS	31.7962	0.9837
	HOTVL	18.9389	0.9374
	ETV	36.2657	0.9967
	OUR	36.8019	0.9973
	TVH1	31.5509	0.9514
Shape2	L0MS	31.6024	0.9463
	HoTVL1	15.6438	0.8137
	ETV	36.6105	0.9972
	OUR	37.6838	0.9975

| Show Table

DownLoad: CSV

Table 2. PSNR and MSSIM of $ T_1 $-weighted brain MRIs with different levels of Gaussian white noises

		$ 0.03 $		$ 0.05 $		$ 0.07 $		$ 0.09 $
		PSNR	MSSIM	PSNR	MSSIM	PSNR	MSSIM	PSNR	MSSIM
	TVH1	26.8509	0.9436	25.3513	0.9176	24.5925	0.8939	23.4850	0.8642
Test	HoTVL1	30.2055	0.9515	27.7438	0.9288	26.4877	0.9069	24.7440	0.8774
Image1	L0MS	29.7718	0.9227	27.8711	0.9088	26.1986	0.8936	24.3809	0.8649
	ETV	32.6699	0.9904	31.1178	0.9844	29.1294	0.9767	28.4238	0.9686
	OUR	33.0513	0.9909	31.2246	0.9846	30.1037	0.9781	28.4666	0.9686
	TVH1	27.4061	0.9230	26.5258	0.8935	25.5879	0.8661	24.0629	0.8348
Test	HoTVL1	30.0791	0.9317	29.0495	0.9065	27.0113	0.8795	25.1211	0.8481
Image2	L0MS	32.0987	0.9246	29.0807	0.9016	27.1344	0.8760	24.9108	0.8382
	ETV	33.4363	0.9911	31.4108	0.9842	29.8199	0.9764	28.6496	0.9698
	OUR	33.9694	0.9916	31.4609	0.9841	29.8766	0.9760	29.0947	0.9703

| Show Table

DownLoad: CSV

Table 3. PSNR and MSSIM of Retinex illusion images

		PSNR	MSSIM
	TVH1	31.7514	0.8887
Checkboard	L0MS	33.1312	0.9602
	HOTVL1	29.5090	0.8725
	ETV	38.7053	0.9936
	OUR	39.2105	0.9934
	TVH1	30.7779	0.9799
	L0MS	30.0935	0.9774
Cube	HOTVL1	27.9950	0.9692
	ETV	29.6898	0.9866
	OUR	33.3660	0.9896

| Show Table

DownLoad: CSV

Related Papers

Cited by

References

[1]	R. Amestoy, E. Provenzi, M. Bertalmío and V. Caselles, A perceptually inspired variational framework for color enhancement, IEEE Transactions on Pattern Analysis and Machine Intelligence, 31 (2009), 458-474.
[2]	M. Benning, F. Knoll, C. Schonlieb and T. Valkonen, Preconditioned ADMM with nonlinear operator constraint, IFIP Conference on System Modeling and Optimization, 494 (2015), 117-126. doi: 10.1007/978-3-319-55795-3_10.
[3]	M. Bertalmío, V. Caselles, E. Provenzi and A. Rizzi, Perceptual color correction through variational techniques, IEEE Transactions on Image Processing, 16 (2007), 1058-1072. doi: 10.1109/TIP.2007.891777.
[4]	M. Bertalmío, V. Caselles and E. Provenzi, Issues about retinex theory and contrast enhancement, International Journal of Computer Vision, 83 (2009), 101-119.
[5]	A. Blake, Boundary conditions for lightness computation in Mondrian world, Computer Vision Graphics Image Processing, 32 (1985), 314-327. doi: 10.1016/0734-189X(85)90054-4.
[6]	S. Boyd, N. Parikh, E. Chu, B. Peleato and J. Eckstein, Distributed optimization and statistical learning via the alternating direction method of multipliers, Foundations and Trends in Machine Learning, 3 (2011), 1-122. doi: 10.1561/9781601984616.
[7]	E. Candes, M. Wakin and S. Boyd, Enhancing sparsity by reweighted $l_1$ minimization, Journal of Fourier Analysis and Applications, 14 (2008), 877-905. doi: 10.1007/s00041-008-9045-x.
[8]	T. Cooper and F. Baqai, Analysis and extensions of the frankle-mccann retinex algorithm, Journal of Electronic Imaging, 13 (2004), 85-93. doi: 10.1117/1.1636182.
[9]	X. Chen, F. Xu and Y. Ye, Lower bound theory of nonzero entries in solutions of $\ell^2-\ell^p$ minimization, SIAM Journal on Scientific Computing, 32 (2010), 2832-2852. doi: 10.1137/090761471.
[10]	X. Chen and W. Zhou, Convergence of the reweighted $\ell_1$ minimization algorithm for $\ell_2-\ell^p$ minimization, Journal Computational Optimization and Application, 59 (2014), 47-61. doi: 10.1007/s10589-013-9553-8.
[11]	J. Douglas and H. Rachford, On the numerical solution of heat conduction problems in two and three space variables, Transactions of the American Mathematical Society, 82 (1956), 421-439. doi: 10.1090/S0002-9947-1956-0084194-4.
[12]	Y. Duan, H. Chang, W. Huang, J. Zhou, Z. Lu and C. Wu, The $L_0$ regularized Mumford-Shah model for bias correction and segmentation of medical images, IEEE Transactions on Image Processing, 24 (2015), 3927-3938. doi: 10.1109/TIP.2015.2451957.
[13]	D. Gabay and B. Mercier, A dual algorithm for the solution of nonlinear variational problems via finite element approximations, Computers and Mathematics with Applications, 2 (1976), 17-40. doi: 10.1016/0898-1221(76)90003-1.
[14]	N. Galatsanos and A. Katsaggelos, Methods for choosing the regularization parameter and estimating the noise variance in image restoration and their relation, IEEE Transactions on Image Processing, 1 (1992), 322-336. doi: 10.1109/83.148606.
[15]	D. Ghilli and K. Kunisch, On monotone and primal-dual active set schemes for $\ell^p$-type problems, $p\in(0, 1]$, Computational Optimizationand Applications, 72 (2019), 45-85. doi: 10.1007/s10589-018-0036-9.
[16]	R. Glowinski, S. Luo and X. Tai, Fast operator-splitting algorithms for variational imaging models: Some recent developments, Handbook of Numerical Analysis, 20 (2019), 191-232.
[17]	R. Glowinski, S. Osher and W. Yin, Splitting Methods in Communication, Imaging, Science, and Engineering, Springer, Cham, 2016. doi: 10.1007/978-3-319-41589-5.
[18]	Z. Gu, F. Li and X. Lv, A detail preserving variational model for image Retinex, Applied Mathematical Modelling, 68 (2019), 643-661. doi: 10.1016/j.apm.2018.11.052.
[19]	P. Hansen and D. Leary, The use of the L-curve in the regularization of discrete ill-posed problems, SIAM Journal on Scientific Computing, 14 (1993), 1487-1503. doi: 10.1137/0914086.
[20]	B. Horn, Determining lightness from an image, Computer Graphics Image Processing, 3 (1974), 277-299. doi: 10.1016/0146-664X(74)90022-7.
[21]	B. Horn, Understanding image intensities, Artificial Intelligence, 8 (1977), 201-231. doi: 10.1016/0004-3702(77)90020-0.
[22]	D. Jobson, Z. Rahman and G. Woodell, Properties and performance of a center/surround retinex, IEEE Transactions on Image Processing, 6 (1997), 451-462. doi: 10.1109/83.557356.
[23]	D. Jobson, Z. Rahman and G. Woodell, A multiscale Retinex for bridging the gap between color image and the human observation of scenes, IEEE Transactions on Image Processing, 6 (2002), 965-976. doi: 10.1109/83.597272.
[24]	Y. Jung, T. Jeong and S. Yun, Non-convex TV denoising corrupted by impulse noise, Inverse Problems and Imaging, 11 (2017), 689-702. doi: 10.3934/ipi.2017032.
[25]	R. Kimmel, M. Elad, D. Shaked, R. Keshet and I. Sobel, A variational framework for retinex, International Journal of Computer Vision, 52 (2003), 7-23.
[26]	M. Lai, Y. Xu and W. Yin, Improved iteratively reweighted least squares for unconstrained smoothed $\ell_q$ minimization, SIAM Journal on Numerical Analysis, 51 (2013), 927-957. doi: 10.1137/110840364.
[27]	E. Land and J. Mccann, Lightness and Retinex theory, Journal of the Optical Society of America, 61 (1971), 1-11.
[28]	E. Land, The Retinex theory of color vision, Scientific American, 237 (1977), 108-128.
[29]	E. Land, An alternative technique for the computation of the designator in the retinex theory of color vision, Proceedings of the National Academy of Sciences, 83 (1986), 3078-3080.
[30]	M. Langer and S. Zucker, Spatially varying illumination: A computational model of converging and diverging sources, European Conference on Computer Vision, 801 (1994), 226-232. doi: 10.1007/BFb0028356.
[31]	A. Lanza1, S. Morigi1 and F. Sgallari, Constrained $TV_p-\ell_2$ model for image restoration, Journal of Scientific Computing, 68 (2016), 64-91. doi: 10.1007/s10915-015-0129-x.
[32]	L. Liu, Z. Pang and Y. Duan, Retinex based on exponent-type total variation scheme, Inverse Problems and Imaging, 12 (2018), 1199-1217. doi: 10.3934/ipi.2018050.
[33]	Z. Liu, C. Wu and Y. Zhao, A new globally convergent algorithm for non-Lipschitz $\ell^p-\ell^q$ minimization, Advances in Computational Mathematics, 45 (2019), 1369-1399. doi: 10.1007/s10444-019-09668-y.
[34]	J. Liang and X. Zhang, Retinex by higher order total variation ${L}^1$ decomposition, Journal of Mathematical Imaging and Vision, 52 (2015), 345-355. doi: 10.1007/s10851-015-0568-x.
[35]	Z. Lu, Iterative reweighted minimization methods for $\ell^p$ regularized unconstrained nonlinear programming, Mathematical Programming: Series A and B, 147 (2014), 277-307. doi: 10.1007/s10107-013-0722-4.
[36]	D. Marini and A. Rizzi, A computational approach to color adaptation effects, Image and Vision Computing, 18 (2000), 1005-1014.
[37]	W. Ma and S. Osher, A TV bregman iterative model of retinex theory, Inverse Problems and Imaging, 6 (2012), 697-708. doi: 10.3934/ipi.2012.6.697.
[38]	J. Mccann, Lessons learned from mondrians applied to real images and color gamuts, Proceedings of the IST/SID 7th Color Imaging Conference, 1999, 1–8.
[39]	J. Morel, A. Petro and C. Sbert, Fast implementation of color constancy algorithms, Proceedings of SPIE, 7241, 2009.
[40]	J. Morel, A. Petro and C. Sbert, A PDE formalization of retinex theory, IEEE Transactions on Image Processing, 19 (2010), 2825-2837. doi: 10.1109/TIP.2010.2049239.
[41]	V. Morozov, Methods for Solving Incorrectly Posed Problems, Springer-Verlag, New York, 1984. doi: 10.1007/978-1-4612-5280-1.
[42]	M. Ng and W. Wang, A total variation model for retinex, SIAM Journal on Imaging Sciences, 4 (2011), 345-365. doi: 10.1137/100806588.
[43]	P. Ochs, A. Dosovitskiy, T. Brox and T. Pock, On iteratively reweighted algorithms for nonsmooth nonconvex optimization in computer vision, SIAM Journal on Imaging Sciences, 8 (2015), 331-372. doi: 10.1137/140971518.
[44]	J. Oliveira, J. Dias and M. Figueiredo, Adaptive total variation image deblurring: A majorizationCminimization approach, Signal Processing, 89 (2009), 1683-1693.
[45]	H. Pan, Y. Wen and H. Zhu, A regularization parameter selection model for total variation based image noise removal, Applied Mathematical Modelling, 68 (2019), 353-367. doi: 10.1016/j.apm.2018.11.032.
[46]	E. Provenzi, D. Marini, L. De Carli and A. Rizzi, Mathematical definition and analysis of the retinex algorithm, Journal of the Optical Society of America A, 22 (2005), 2613-2621. doi: 10.1364/JOSAA.22.002613.
[47]	S. Sabacha and M. Teboulle, Lagrangian methods for composite optimization, Handbook of Numerical Analysis, 20 (2019), 401-436.
[48]	A. Theljani and K. Chen, A Nash game based variational model for joint image intensity correction and registration to deal with varying illumination, Inverse Problems, 36 (2020), 034002.
[49]	Y. Wen and R. Chan, Using generalized cross validation to select regularization parameter for total variation regularization problems, Inverse Problems and Imaging, 12 (2018), 1103-1120. doi: 10.3934/ipi.2018046.
[50]	W. Wang and C. He, A variational model with barrier functionals for Retinex, SIAM Journal on Imaging Sciences, 8 (2015), 1955-1980. doi: 10.1137/15M1006908.
[51]	J. Zhang, R. Chen, C. Deng and S. Wang, Fast linearized augmented method for Euler's elastica model, Numerical Mathematics:Theory, Methods and Applications, 10 (2017), 98-115. doi: 10.4208/nmtma.2017.m1611.
[52]	X. Zhang, Y. Shi, Z. Pang and Y. Zhu, Fast algorithm for image denoising with different boundary conditions, Journal of the Franklin Institute, 354 (2017), 4595-4614. doi: 10.1016/j.jfranklin.2017.04.011.
[53]	D. Zosso, G. Tran and S. Osher, A unifying retinex model based on non-local differential operators, Computational Imaging XI, 865702, 2013..
[54]	W. Zuo, D. Meng, L. Zhang, X. Feng and D. Zhang, A generalized iterated shrinkage algorithm for non-convex sparse coding, IEEE International Conference on Computer Vision, 2013,217–224.