A variational saturation-value model for image decomposition: Illumination and reflectance

Wei Wang; Caifei Li

doi:10.3934/ipi.2021061

Article Contents

2022, Volume 16, Issue 3: 547-567. Doi: 10.3934/ipi.2021061

This issue Previous Article Generative imaging and image processing via generative encoder Next Article Uniqueness and numerical reconstruction for inverse problems dealing with interval size search

A variational saturation-value model for image decomposition: Illumination and reflectance

Wei Wang^, and
Caifei Li

School of Mathematical Sciences, Tongji University, Shanghai, China

^*Corresponding author: Wei Wang
^*Corresponding author: Wei Wang

Received: December 31, 2020

Revised: June 30, 2021

Published: June 2022

Wei Wang is supported by Natural Science Foundation of Shanghai (18ZR1441800)

Abstract Full Text(HTML) Figure(11) / Table(3) Related Papers Cited by

Abstract

In this paper, we study to decompose a color image into the illumination and reflectance components in saturation-value color space. By considering the spatial smoothness of the illumination component, the total variation regularization of the reflectance component, and the data-fitting in saturation-value color space, we develop a novel variational saturation-value model for image decomposition. The main aim of the proposed model is to formulate the decomposition of a color image such that the illumination component is uniform with only brightness information, and the reflectance component contains the color information. We establish the theoretical result about the existence of the solution of the proposed minimization problem. We employ a primal-dual algorithm to solve the proposed minimization problem. Experimental results are shown to illustrate the effectiveness of the proposed decomposition model in saturation-value color space, and demonstrate the performance of the proposed method is better than the other testing methods.

Keywords:

Mathematics Subject Classification: 68U10, 65K10, 65J22, 90C25.

Citation:

Full Text(HTML)

Figure 1. Results by using different pairs of $ \mu $ and $ \lambda $. The first row is the original image; the second and fourth rows are the reflectance components; the third and the fifth rows are the illumination components

Download: Full-size image PowerPoint slide

Figure 2. The first column: the original image; the second column: the reflectance components; the third column: the illumination components. From top to bottom: the results by using the proposed model, L1-Ma model, TV-Ma model, TV-Ng-HSV model, TV-Ng-RGB model, Dictionary model, Multigrid model respectively

Download: Full-size image PowerPoint slide

Figure 3. The first column: the original image; the second column: the reflectance components; the third column: the illumination components. From top to bottom: the results by using the proposed model, L1-Ma model, TV-Ma model, TV-Ng-HSV model, TV-Ng-RGB model, Dictionary model, Multigrid model respectively

Download: Full-size image PowerPoint slide

Figure 4. The first column: the original image; the second column: the reflectance components; the third column: the illumination components. From top to bottom: the results by using the proposed model, L1-Ma model, TV-Ma model, TV-Ng-HSV model, TV-Ng-RGB model, Dictionary model, Multigrid model respectively

Download: Full-size image PowerPoint slide

Figure 5. The first column: the original image; the second column: the reflectance components; the third column: the illumination components. From top to bottom: the results by using the proposed model, L1-Ma model, TV-Ma model, TV-Ng-HSV model, TV-Ng-RGB model, Dictionary model, Multigrid model respectively

Download: Full-size image PowerPoint slide

Figure 6. Adelson's checker shadow illusion example. From top to bottom: the results by using the proposed SV model, L1-Ma model, TV-Ma model, TV-Ng-HSV model, TV-Ng-RGB model, Dictionary method, Multigrid method respectively

Download: Full-size image PowerPoint slide

Figure 7. Deer example. (a): the original image; (b): the bias image; (c) - (i): the reflectance results by using the proposed SV model, L1-Ma model, TV-Ma model, TV-Ng-HSV model, TV-Ng-RGB model, Dictionary method, Multigrid method respectively

Download: Full-size image PowerPoint slide

Figure 8. Parthenon example. (a): the original image; (b): the bias image; (c) - (i): the reflectance results by using the proposed SV model, L1-Ma model, TV-Ma model, TV-Ng-HSV model, TV-Ng-RGB model, Dictionary method, Multigrid method respectively

Download: Full-size image PowerPoint slide

Figure 9. Clay figure example. (a): the original image; (b): the bias image; (c) - (i): the reflectance results by using the proposed SV model, L1-Ma model, TV-Ma model, TV-Ng-HSV model, TV-Ng-RGB model, Dictionary method, Multigrid method respectively

Download: Full-size image PowerPoint slide

Figure 10. Bridge example. (a): the original image; (b): the bias image; (c) - (i): the reflectance results by using the proposed SV model, L1-Ma model, TV-Ma model, TV-Ng-HSV model, TV-Ng-RGB model, Dictionary method, Multigrid method respectively

Download: Full-size image PowerPoint slide

Figure 11. Soldiers example. (a): the original image; (b): the bias image; (c) - (i): the reflectance results by using the proposed SV model, L1-Ma model, TV-Ma model, TV-Ng-HSV model, TV-Ng-RGB model, Dictionary method, Multigrid method respectively

Download: Full-size image PowerPoint slide

Algorithm A:
   Step1: Choose stopping criteria $ \epsilon $ and initialization $ {\bf{u}}^{0} $.
   Step2: For fixed $ {\bf{u}}^{n} $, update $ {\bf{\bar{u}}} $ by solving:
$ \begin{equation} (8) \;\;\;\;\;\;\;\; \max\limits_{{\bf{\bar{u}}}} \left\lbrace -<{\bf{u}}^{n}, \text{div}{\bf{\bar{u}}}> - f^{*}({\bf{\bar{u}}}) - \frac{1}{2\sigma}\left\| {\bf{\bar{u}}} - {\bf{\bar{u}}}^{n}\right\| _{2}^{2}\right\rbrace. \end{equation}$
   Step3: For fixed $ {\bf{\bar{u}}}^{n+1} $, update $ {\bf{u}} $ by solving:
$ \begin{equation} (9)\;\;\;\;\;\;\;\;\;\; \min\limits_{{\bf{u}}} \left\lbrace -<{\bf{u}},\text{div}{\bf{\bar{u}}}^{n+1}> + g({\bf{u}}) - \frac{1}{2\tau}\left\| {\bf{u}} - {\bf{u}}^{n}\right\|_{2}^{2} \right\rbrace. \end{equation} $
   Step4: Iterate until $ \frac{\parallel{\bf{u}}^{n+1}-{\bf{u}}^{n}\parallel^2}{\parallel{\bf{u}}^{n}\parallel^2}\leq \epsilon $.

| Show Table

DownLoad: CSV

Algorithm B:
   Step1: Choose step size $ \sigma > 0 $ and $ \tau >0 $. Initialize $ {\bf{u}}^{0} $ and $ {\bf{\bar{u}}}^{0} $.
   Step2: For each iteration:
$\begin{equation} (14)\;\;\;\;\;\;\;\; \left\lbrace \begin{aligned} {\bf{\bar{u}}}^{n+1} & = {\bf{prox}}_{f^{*}}\left( {\bf{\bar{u}}}^{n} + \sigma\bigtriangledown{\bf{u}}^{n}\right),\\ {\bf{u}}^{n+1} & = {\bf{prox}}_{g}\left( {\bf{u}}^{n} - \tau \text{div}{\bf{\bar{u}}}^{n+1}\right). \end{aligned} \right. \end{equation} $
   Step3: Repeat $ {\bf{Step 2}} $ until $ \frac{\parallel{\bf{u}}^{n+1}-{\bf{u}}^{n}\parallel^2}{\parallel{\bf{u}}^{n}\parallel^2}\leq \epsilon $.

| Show Table

DownLoad: CSV

Table 1. SSIM, PSNR and S-CIELAB color error between original images and reflectance images

SSIM	original image
SSIM	SV	TV-Ma	L1-Ma	TV-Ng-HSV	TV-Ng-RGB	Dictionary	Multigrid
deer	0.8804	0.4212	0.6019	0.9098	0.6017	0.3684	0.8634
Parthenon	0.8476	0.6347	0.6036	0.7735	0.6996	0.7733	0.7932
clay figure	0.8703	0.1671	0.8666	0.8569	0.6425	0.4150	0.9051
bridge	0.8646	0.6285	0.6645	0.7508	0.5885	0.8166	0.8349
soldiers	0.8564	0.2343	0.8289	0.7625	0.5295	0.6100	0.8041
PSNR	original image
PSNR	SV	TV-Ma	L1-Ma	TV-Ng-HSV	TV-Ng-RGB	Dictionary	Multigrid
deer	19.5951	17.2721	18.5804	15.5689	8.5341	9.5946	13.2474
Parthenon	17.5351	17.0368	13.1229	10.9352	8.5158	13.0687	11.5762
clay figure	18.5353	15.0454	15.5129	13.2862	11.1804	8.9712	15.6808
bridge	17.9966	18.1970	18.6187	9.9868	8.4870	14.0895	12.1009
soldiers	17.3405	15.3468	18.9972	10.7686	7.8472	11.8415	11.7106
S-CIELAB error	original image
S-CIELAB error	SV	TV-Ma	L1-Ma	TV-Ng-HSV	TV-Ng-RGB	Dictionary	Multigrid
deer	4.87%	25.46%	10.47%	7.24%	91.33%	82.33%	24.13%
Parthenon	3.03%	8.54%	36.89%	44.82%	62.12%	37.75%	43.87%
clay figure	10.07%	77.20%	15.42%	31.47%	67.22%	92.31%	15.37%
bridge	2.63%	16.08%	3.89%	71.92%	81.82%	29.05%	31.93%
soldiers	11.00%	78.01%	3.89%	63.79%	94.14%	75.67%	55.22%

| Show Table

DownLoad: CSV

Related Papers

Cited by

References

[1]	P. Arbelaez, M. Maire, C. Fowlkes and J. Malik, Contour detection and hierarchical image segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 33 (2011), 898-916.
[2]	T. Batard, Heat equations on vector bundles - application to color image regularization, J. Math. Imaging Vision, 41 (2011), 59-85. doi: 10.1007/s10851-011-0265-3.
[3]	M. Bertalmio and J. D. Cowan, Implementing the retinex algorithm with wilson-cowan equations, Journal of Physiology Paris, 103 (2009), 69-72.
[4]	A. Blakea, Boundary conditions for lightness computation in mondrian world, Computer Vision, Graphics and Image Processing, 32 (1985), 314-327.
[5]	A. Chambolle and T. Pock, A first-order primal-dual algorithm for convex problems with applications to imaging, J. Math. Imaging Vision, 40 (2011), 120-145. doi: 10.1007/s10851-010-0251-1.
[6]	H. Chang, M. K. Ng, W. Wang and T. Zeng, Retinex image enhancement via a learned dictionary, Optical Engineering, 54 (2015), 013107.
[7]	P. Denis, P. Carre and C. Fernandez-Maloigne, Spatial and spectral quaternionic approaches for colour images, Computer Vision and Image Understanding, 107 (2007), 74-87.
[8]	M. Elad, Retinex by two bilateral filters, Lecture Notes in Computer Science, 3459 (2005), 217-229.
[9]	E. Esser, X. Zhang and T. F. Chan, A general framework for a class of first order primal-dual algorithms for convex optimization in imaging science, SIAM J. Imaging Sci., 3 (2010), 1015-1046. doi: 10.1137/09076934X.
[10]	J. Frankle and J. McCann, Method and apparatus for lightness imaging, US Patent, 1983.
[11]	B. Funt, F. Ciuera and J. McCann, Retinex in matlab, Journal of Electronic Imaging, 13 (2004), 48-57.
[12]	B. Funt, M. Drew and M. Brockington, Recovering shading from color images, Lecture Notes in Computer Science, 588 (1992), 124-132.
[13]	R. C. Gonzales and R. E. Woods, Digital Image Processing, 3$^{nd}$ edition, Pearson International Edition, Prentice Hall, 2008.
[14]	W. R. Hamilton, Elements of Quaternions, Longmans, Green and Co., London, 1866.
[15]	B. K. P. Horn, Determining lightness from an image, Computer Graphics and Image Processing, 3 (1974), 277-299.
[16]	Z. Jia, M. K. Ng and W. Wang, Color image restoration by saturation-value total variation, SIAM J. Imaging Sci, 12 (2019), 972-1000. doi: 10.1137/18M1230451.
[17]	D. J. Jobson, Z. Rahman and G. A. Woodell, Properties and performance of the center/surround retinex, IEEE Transactions on Image Processing, 6 (1997), 451-462.
[18]	D. J. Jobson, Z. Rahman and G. A. Woodell, A multiscale Retinex for bridging the gap between color image and the human observation of scenes, IEEE Transactions on Image Processing, 6 (1997), 965-976.
[19]	R. Kimmel, M. Elad, D. Shaked, R. Keshet and I. Sobel, A variational framework for Retinex, International Journal of Computer Vision, 52 (2003), 7-23.
[20]	E. H. Land and J. J. McCann, Lightness and retinex theory, Journal of the Optical Society of America, 61 (1971), 1-11.
[21]	E. H. Land, The retinex theory of color vision, Scientific American, 237 (1977), 108-128.
[22]	E. H. Land, Recent advances in the retinex theory and some implications for cortical computations: Color vision and natural image, Proceedings of the National Academy of Sciences of the United States of America, 80 (1983), 5163-5169.
[23]	N. Limare, A.-B. Petro, C. Sbert and J.-M. Morel, Retinex poisson equation: A model for color perception, Image Processing On Line, 1 (2011), 39-50.
[24]	W. Ma and S. Osher, A TV Bregman iterative model of retinex theory, Inverse Probl. Imaging, 6 (2012), 697-708. doi: 10.3934/ipi.2012.6.697.
[25]	W. Ma, J.-M. Morel, S. Osher and A. Chien, An L1-based variational model for Retinex theory and its application to medical images, IEEE Computer Society Conference on Computer Vision and Pattern Recognition, (2011), 153–160.
[26]	J.-M. Morel, A. B. Petro and C. Sbert, Fast implementation of color constancy algorithms, Proceedings of the SPIE, 7241 (2009).
[27]	J. M. Morel, A. B. Petro and C. Sbert, A PDE formalization of retinex theory, IEEE Transactions on Image Processing, 19 (2010), 2825-2837. doi: 10.1109/TIP.2010.2049239.
[28]	M. K. Ng and W. Wang, A total variation model for retinex, SIAM J. Imaging Sci., 4 (2011), 345-365. doi: 10.1137/100806588.
[29]	Z.-F. Pang, Y.-M. Zhou, T. Wu and D.-J. Li, Image denoising via a new anisotropic total-variation-based model, Signal Processing: Image Communication, 74 2019,140–152.
[30]	L. Sun and Y.-M. Huang, A modulus-based multigrid method for image retinex, Appl. Numer. Math., 164 (2021), 199-210. doi: 10.1016/j.apnum.2020.11.011.
[31]	D. Terzopoulos, Image analysis using multigrid relaxation method, IEEE Transactions on Pattern Analysis and Machine Intelligence, 8 (1986), 129-139.
[32]	T. Wu, X. Gu, Y. Wang and T. Zeng, Adaptive total variation based image segmentation with semi-proximal alternating minimization, Signal Processing, 183 (2021).
[33]	X. Zhang and B. A. Wandell, A spatial extension of CIELAB for digital color image reproduction, Journal of the Society for Information Display, 5 (1997), 61-63.