Article Contents
Article Contents

# A variational saturation-value model for image decomposition: Illumination and reflectance

• *Corresponding author: Wei Wang

Wei Wang is supported by Natural Science Foundation of Shanghai (18ZR1441800)

• In this paper, we study to decompose a color image into the illumination and reflectance components in saturation-value color space. By considering the spatial smoothness of the illumination component, the total variation regularization of the reflectance component, and the data-fitting in saturation-value color space, we develop a novel variational saturation-value model for image decomposition. The main aim of the proposed model is to formulate the decomposition of a color image such that the illumination component is uniform with only brightness information, and the reflectance component contains the color information. We establish the theoretical result about the existence of the solution of the proposed minimization problem. We employ a primal-dual algorithm to solve the proposed minimization problem. Experimental results are shown to illustrate the effectiveness of the proposed decomposition model in saturation-value color space, and demonstrate the performance of the proposed method is better than the other testing methods.

Mathematics Subject Classification: 68U10, 65K10, 65J22, 90C25.

 Citation:

• Figure 1.  Results by using different pairs of $\mu$ and $\lambda$. The first row is the original image; the second and fourth rows are the reflectance components; the third and the fifth rows are the illumination components

Figure 2.  The first column: the original image; the second column: the reflectance components; the third column: the illumination components. From top to bottom: the results by using the proposed model, L1-Ma model, TV-Ma model, TV-Ng-HSV model, TV-Ng-RGB model, Dictionary model, Multigrid model respectively

Figure 3.  The first column: the original image; the second column: the reflectance components; the third column: the illumination components. From top to bottom: the results by using the proposed model, L1-Ma model, TV-Ma model, TV-Ng-HSV model, TV-Ng-RGB model, Dictionary model, Multigrid model respectively

Figure 4.  The first column: the original image; the second column: the reflectance components; the third column: the illumination components. From top to bottom: the results by using the proposed model, L1-Ma model, TV-Ma model, TV-Ng-HSV model, TV-Ng-RGB model, Dictionary model, Multigrid model respectively

Figure 5.  The first column: the original image; the second column: the reflectance components; the third column: the illumination components. From top to bottom: the results by using the proposed model, L1-Ma model, TV-Ma model, TV-Ng-HSV model, TV-Ng-RGB model, Dictionary model, Multigrid model respectively

Figure 6.  Adelson's checker shadow illusion example. From top to bottom: the results by using the proposed SV model, L1-Ma model, TV-Ma model, TV-Ng-HSV model, TV-Ng-RGB model, Dictionary method, Multigrid method respectively

Figure 7.  Deer example. (a): the original image; (b): the bias image; (c) - (i): the reflectance results by using the proposed SV model, L1-Ma model, TV-Ma model, TV-Ng-HSV model, TV-Ng-RGB model, Dictionary method, Multigrid method respectively

Figure 8.  Parthenon example. (a): the original image; (b): the bias image; (c) - (i): the reflectance results by using the proposed SV model, L1-Ma model, TV-Ma model, TV-Ng-HSV model, TV-Ng-RGB model, Dictionary method, Multigrid method respectively

Figure 9.  Clay figure example. (a): the original image; (b): the bias image; (c) - (i): the reflectance results by using the proposed SV model, L1-Ma model, TV-Ma model, TV-Ng-HSV model, TV-Ng-RGB model, Dictionary method, Multigrid method respectively

Figure 10.  Bridge example. (a): the original image; (b): the bias image; (c) - (i): the reflectance results by using the proposed SV model, L1-Ma model, TV-Ma model, TV-Ng-HSV model, TV-Ng-RGB model, Dictionary method, Multigrid method respectively

Figure 11.  Soldiers example. (a): the original image; (b): the bias image; (c) - (i): the reflectance results by using the proposed SV model, L1-Ma model, TV-Ma model, TV-Ng-HSV model, TV-Ng-RGB model, Dictionary method, Multigrid method respectively

 Algorithm A:    Step1: Choose stopping criteria $\epsilon$ and initialization ${\bf{u}}^{0}$.    Step2: For fixed ${\bf{u}}^{n}$, update ${\bf{\bar{u}}}$ by solving: $$$(8) \;\;\;\;\;\;\;\; \max\limits_{{\bf{\bar{u}}}} \left\lbrace -<{\bf{u}}^{n}, \text{div}{\bf{\bar{u}}}> - f^{*}({\bf{\bar{u}}}) - \frac{1}{2\sigma}\left\| {\bf{\bar{u}}} - {\bf{\bar{u}}}^{n}\right\| _{2}^{2}\right\rbrace.$$$    Step3: For fixed ${\bf{\bar{u}}}^{n+1}$, update ${\bf{u}}$ by solving: $$$(9)\;\;\;\;\;\;\;\;\;\; \min\limits_{{\bf{u}}} \left\lbrace -<{\bf{u}},\text{div}{\bf{\bar{u}}}^{n+1}> + g({\bf{u}}) - \frac{1}{2\tau}\left\| {\bf{u}} - {\bf{u}}^{n}\right\|_{2}^{2} \right\rbrace.$$$    Step4: Iterate until $\frac{\parallel{\bf{u}}^{n+1}-{\bf{u}}^{n}\parallel^2}{\parallel{\bf{u}}^{n}\parallel^2}\leq \epsilon$.
 Algorithm B:    Step1: Choose step size $\sigma > 0$ and $\tau >0$. Initialize ${\bf{u}}^{0}$ and ${\bf{\bar{u}}}^{0}$.    Step2: For each iteration: (14)\;\;\;\;\;\;\;\; \left\lbrace \begin{aligned} {\bf{\bar{u}}}^{n+1} & = {\bf{prox}}_{f^{*}}\left( {\bf{\bar{u}}}^{n} + \sigma\bigtriangledown{\bf{u}}^{n}\right),\\ {\bf{u}}^{n+1} & = {\bf{prox}}_{g}\left( {\bf{u}}^{n} - \tau \text{div}{\bf{\bar{u}}}^{n+1}\right). \end{aligned} \right.    Step3: Repeat ${\bf{Step 2}}$ until $\frac{\parallel{\bf{u}}^{n+1}-{\bf{u}}^{n}\parallel^2}{\parallel{\bf{u}}^{n}\parallel^2}\leq \epsilon$.

Table 1.  SSIM, PSNR and S-CIELAB color error between original images and reflectance images

 SSIM original image SV TV-Ma L1-Ma TV-Ng-HSV TV-Ng-RGB Dictionary Multigrid deer 0.8804 0.4212 0.6019 0.9098 0.6017 0.3684 0.8634 Parthenon 0.8476 0.6347 0.6036 0.7735 0.6996 0.7733 0.7932 clay figure 0.8703 0.1671 0.8666 0.8569 0.6425 0.4150 0.9051 bridge 0.8646 0.6285 0.6645 0.7508 0.5885 0.8166 0.8349 soldiers 0.8564 0.2343 0.8289 0.7625 0.5295 0.6100 0.8041 PSNR original image SV TV-Ma L1-Ma TV-Ng-HSV TV-Ng-RGB Dictionary Multigrid deer 19.5951 17.2721 18.5804 15.5689 8.5341 9.5946 13.2474 Parthenon 17.5351 17.0368 13.1229 10.9352 8.5158 13.0687 11.5762 clay figure 18.5353 15.0454 15.5129 13.2862 11.1804 8.9712 15.6808 bridge 17.9966 18.1970 18.6187 9.9868 8.4870 14.0895 12.1009 soldiers 17.3405 15.3468 18.9972 10.7686 7.8472 11.8415 11.7106 S-CIELAB error original image SV TV-Ma L1-Ma TV-Ng-HSV TV-Ng-RGB Dictionary Multigrid deer 4.87% 25.46% 10.47% 7.24% 91.33% 82.33% 24.13% Parthenon 3.03% 8.54% 36.89% 44.82% 62.12% 37.75% 43.87% clay figure 10.07% 77.20% 15.42% 31.47% 67.22% 92.31% 15.37% bridge 2.63% 16.08% 3.89% 71.92% 81.82% 29.05% 31.93% soldiers 11.00% 78.01% 3.89% 63.79% 94.14% 75.67% 55.22%
•  [1] P. Arbelaez, M. Maire, C. Fowlkes and J. Malik, Contour detection and hierarchical image segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 33 (2011), 898-916. [2] T. Batard, Heat equations on vector bundles - application to color image regularization, J. Math. Imaging Vision, 41 (2011), 59-85.  doi: 10.1007/s10851-011-0265-3. [3] M. Bertalmio and J. D. Cowan, Implementing the retinex algorithm with wilson-cowan equations, Journal of Physiology Paris, 103 (2009), 69-72. [4] A. Blakea, Boundary conditions for lightness computation in mondrian world, Computer Vision, Graphics and Image Processing, 32 (1985), 314-327. [5] A. Chambolle and T. Pock, A first-order primal-dual algorithm for convex problems with applications to imaging, J. Math. Imaging Vision, 40 (2011), 120-145.  doi: 10.1007/s10851-010-0251-1. [6] H. Chang, M. K. Ng, W. Wang and T. Zeng, Retinex image enhancement via a learned dictionary, Optical Engineering, 54 (2015), 013107. [7] P. Denis, P. Carre and C. Fernandez-Maloigne, Spatial and spectral quaternionic approaches for colour images, Computer Vision and Image Understanding, 107 (2007), 74-87. [8] M. Elad, Retinex by two bilateral filters, Lecture Notes in Computer Science, 3459 (2005), 217-229. [9] E. Esser, X. Zhang and T. F. Chan, A general framework for a class of first order primal-dual algorithms for convex optimization in imaging science, SIAM J. Imaging Sci., 3 (2010), 1015-1046.  doi: 10.1137/09076934X. [10] J. Frankle and J. McCann, Method and apparatus for lightness imaging, US Patent, 1983. [11] B. Funt, F. Ciuera and J. McCann, Retinex in matlab, Journal of Electronic Imaging, 13 (2004), 48-57. [12] B. Funt, M. Drew and M. Brockington, Recovering shading from color images, Lecture Notes in Computer Science, 588 (1992), 124-132. [13] R. C. Gonzales and R. E. Woods, Digital Image Processing, 3$^{nd}$ edition, Pearson International Edition, Prentice Hall, 2008. [14] W. R. Hamilton, Elements of Quaternions, Longmans, Green and Co., London, 1866. [15] B. K. P. Horn, Determining lightness from an image, Computer Graphics and Image Processing, 3 (1974), 277-299. [16] Z. Jia, M. K. Ng and W. Wang, Color image restoration by saturation-value total variation, SIAM J. Imaging Sci, 12 (2019), 972-1000.  doi: 10.1137/18M1230451. [17] D. J. Jobson, Z. Rahman and G. A. Woodell, Properties and performance of the center/surround retinex, IEEE Transactions on Image Processing, 6 (1997), 451-462. [18] D. J. Jobson, Z. Rahman and G. A. Woodell, A multiscale Retinex for bridging the gap between color image and the human observation of scenes, IEEE Transactions on Image Processing, 6 (1997), 965-976. [19] R. Kimmel, M. Elad, D. Shaked, R. Keshet and I. Sobel, A variational framework for Retinex, International Journal of Computer Vision, 52 (2003), 7-23. [20] E. H. Land and J. J. McCann, Lightness and retinex theory, Journal of the Optical Society of America, 61 (1971), 1-11. [21] E. H. Land, The retinex theory of color vision, Scientific American, 237 (1977), 108-128. [22] E. H. Land, Recent advances in the retinex theory and some implications for cortical computations: Color vision and natural image, Proceedings of the National Academy of Sciences of the United States of America, 80 (1983), 5163-5169. [23] N. Limare, A.-B. Petro, C. Sbert and J.-M. Morel, Retinex poisson equation: A model for color perception, Image Processing On Line, 1 (2011), 39-50. [24] W. Ma and S. Osher, A TV Bregman iterative model of retinex theory, Inverse Probl. Imaging, 6 (2012), 697-708.  doi: 10.3934/ipi.2012.6.697. [25] W. Ma, J.-M. Morel, S. Osher and A. Chien, An L1-based variational model for Retinex theory and its application to medical images, IEEE Computer Society Conference on Computer Vision and Pattern Recognition, (2011), 153–160. [26] J.-M. Morel, A. B. Petro and C. Sbert, Fast implementation of color constancy algorithms, Proceedings of the SPIE, 7241 (2009). [27] J. M. Morel, A. B. Petro and C. Sbert, A PDE formalization of retinex theory, IEEE Transactions on Image Processing, 19 (2010), 2825-2837.  doi: 10.1109/TIP.2010.2049239. [28] M. K. Ng and W. Wang, A total variation model for retinex, SIAM J. Imaging Sci., 4 (2011), 345-365.  doi: 10.1137/100806588. [29] Z.-F. Pang, Y.-M. Zhou, T. Wu and D.-J. Li, Image denoising via a new anisotropic total-variation-based model, Signal Processing: Image Communication, 74 2019,140–152. [30] L. Sun and Y.-M. Huang, A modulus-based multigrid method for image retinex, Appl. Numer. Math., 164 (2021), 199-210.  doi: 10.1016/j.apnum.2020.11.011. [31] D. Terzopoulos, Image analysis using multigrid relaxation method, IEEE Transactions on Pattern Analysis and Machine Intelligence, 8 (1986), 129-139. [32] T. Wu, X. Gu, Y. Wang and T. Zeng, Adaptive total variation based image segmentation with semi-proximal alternating minimization, Signal Processing, 183 (2021). [33] X. Zhang and B. A. Wandell, A spatial extension of CIELAB for digital color image reproduction, Journal of the Society for Information Display, 5 (1997), 61-63.

Figures(11)

Tables(3)