# American Institute of Mathematical Sciences

August  2020, 14(4): 733-755. doi: 10.3934/ipi.2020034

## Saturation-Value Total Variation model for chromatic aberration correction

 1 School of Mathematical Sciences, Tongji University, Shanghai, China 2 School of Mathematical Sciences, Shanghai Jiao Tong University, Shanghai, China 3 Department of Mathematics, The University of Hong Kong, Pokfulam, Hong Kong

* Corresponding author: Ling Pi

Received  September 2019 Revised  February 2020 Published  May 2020

Fund Project: Wei Wang is supported by Natural Science Foundation of Shanghai and Fundamental Research Funds for the Central Universities of China (22120180255, 22120180067), Michael K. Ng is supported in part by the HKRGC GRF 12306616, 12200317, 12300218 and 12300519

Chromatic aberration generally occurs in the regions of sharp edges in captured digital color images. The main aim of this paper is to propose and develop a novel Saturation-Value Total Variation (SVTV) model for chromatic aberration correction. In the proposed optimization model, there are three terms for the correction purpose. The SVTV regularization term is to model the target color image in HSV color space instead of RGB color space, and to avoid oscillations in the recovering process. In correction process, the gradient matching terms based on the green component are used to govern both red and blue components, and the intensity terms are employed to fit red, green and blue data components. The existence of the minimizer of the optimization model is analyzed and an efficient optimization algorithm is also developed for solving the resulting variational problem. Experimental results are presented to illustrate the effectiveness of the proposed model and to show that the correction results are better than those by using the other testing methods.

Citation: Wei Wang, Ling Pi, Michael K. Ng. Saturation-Value Total Variation model for chromatic aberration correction. Inverse Problems & Imaging, 2020, 14 (4) : 733-755. doi: 10.3934/ipi.2020034
##### References:
 [1] G. Aubert and P. Kornprobst, Mathematical Problems in Image Processing, Springer-Verlag, New York, 2002.  Google Scholar [2] B. E. Bayer, Color Imaging Array, U.S. Patent No. 3971065, 1976. Google Scholar [3] T. E. Boult and G. Wolberg, Correcting chromatic aberrations using image warping, in Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1992,684–687. Google Scholar [4] J. Chang, H. Kang and M. G. Kang, Correction of axial and lateral chromatic aberration with false color filtering, IEEE Trans. Image Process., 22 (2013), 1186-1198.  doi: 10.1109/TIP.2012.2228489.  Google Scholar [5] D. Gabay and B. Mercier, A dual algorithm for the solution of nonlinear variational problems via finite element approximation, Comput. Math. Appl., 2 (1976), 17-40.   Google Scholar [6] N. A. Ibraheem, M. M. Hasan, R. Z. Khan and P. K. Mishra, Understanding color models: A review, ARPN Journal of Science and Technology, 2 (2012), 265-275.   Google Scholar [7] Z. Jia, M. K. Ng and W. Wang, Color image restoration by saturation-value (SV) total variation, SIAM J. Imaging Sci., 12 (2019), 972-1000.  doi: 10.1137/18M1230451.  Google Scholar [8] S. B. Kang, Automatic removal of chromatic aberration from a single image, in 2007 IEEE Conference on Computer Vision and Pattern Recognition, 2007, 1–8. Google Scholar [9] H. Kang, S.-H. Lee, J. Chang and M. G. Kang, Partial differential equation-based approach for removal of chromatic aberration with local characteristics, J. of Electronic Imaging, 19 (2010), 1-8.   Google Scholar [10] V. Kaufmann and R. Ladstädter, Elimination of color fringes in digital photographs caused by lateral chromatic aberration, in Proc. 20th Int. Comm. Int. Photogramm. Archit. Symp. Conf., Oct., 2005, 1–6. Google Scholar [11] H. K. Kelda and P. Kaur, A review: Color models in image processing, Int. J. Computer Technology and Applications, 5 (2014), 319-322.   Google Scholar [12] B. Kim and R. Park, Automatic detection and correction of purple fringing using the gradient information and desaturation, in Proc. 16th Eur. Signal Process. Conf., Oct., 2008, 1–5. Google Scholar [13] R. Kimmel, Demosaicing: Image reconstruction from color CCD samples, IEEE Trans. Image Process., 8 (1999), 1221-1228.   Google Scholar [14] H. Kour, Analysis on image color model, International Journal of Advanced Research in Computer and Communication Engineering, 4 (2015), 1-3.   Google Scholar [15] P. B. Kruger, S. Mathews, K. R. Aggarwala and N. Sanchez, Chromatic aberration and ocular focus: Fincham revisited, Vision Research, 33 (1993), 1397-1411.  doi: 10.1016/0042-6989(93)90046-Y.  Google Scholar [16] J. Mallon and P. F. Whelan, Calibration and removal of lateral chromatic aberration in images, Pattern Recognition Letters, 28 (2007), 125-135.  doi: 10.1016/j.patrec.2006.06.013.  Google Scholar [17] D. H. Marimont and B. A. Wandell, Matching color images: The effects of axial chromatic aberration, Journal of the Optical Society of America A, 11 (1994), 3113-3122.  doi: 10.1364/JOSAA.11.003113.  Google Scholar [18] D. Martin, C. Fowlkes, D. Tal and J. Malik, A Database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics, Proceedings of IEEE International Conference on Computer Vision (ICCV) 2001, 2 (2001), 416-423.  doi: 10.1109/ICCV.2001.937655.  Google Scholar [19] P. Mouroulis and J. Macdonald, Geometrical Optics and Optical Design, 1$^{st}$ edition, Oxford U. Press, New York, 1997.   Google Scholar [20] L. Pi, W. Wang and M. Ng, A spatially variant total variational model for chromatic aberration correction, Journal of Visual Communication and Image Representation, 41 (2016), 296-304.  doi: 10.1016/j.jvcir.2016.10.009.  Google Scholar [21] L. N. Thibos, A. Bradley, D. L. Still, X. Zhang and P. A. Howarth, Theory and measurement of ocular chromatic aberration, Vision Research, 30 (1990), 33-49.  doi: 10.1016/0042-6989(90)90126-6.  Google Scholar [22] R. G. Willson and S. A. Shafer, Active lens control for high precision computer imaging, IEEE International Conference on Robotics and Automation, 3 (1991), 2063-2070.  doi: 10.1109/ROBOT.1991.131931.  Google Scholar [23] X. Zhang and B. A. Wandell, A spatial extension of CIELAB for digital color-image reproduction, Journal of the Society for Information Display, 5 (1997), 61-63.  doi: 10.1889/1.1985127.  Google Scholar

show all references

##### References:
 [1] G. Aubert and P. Kornprobst, Mathematical Problems in Image Processing, Springer-Verlag, New York, 2002.  Google Scholar [2] B. E. Bayer, Color Imaging Array, U.S. Patent No. 3971065, 1976. Google Scholar [3] T. E. Boult and G. Wolberg, Correcting chromatic aberrations using image warping, in Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1992,684–687. Google Scholar [4] J. Chang, H. Kang and M. G. Kang, Correction of axial and lateral chromatic aberration with false color filtering, IEEE Trans. Image Process., 22 (2013), 1186-1198.  doi: 10.1109/TIP.2012.2228489.  Google Scholar [5] D. Gabay and B. Mercier, A dual algorithm for the solution of nonlinear variational problems via finite element approximation, Comput. Math. Appl., 2 (1976), 17-40.   Google Scholar [6] N. A. Ibraheem, M. M. Hasan, R. Z. Khan and P. K. Mishra, Understanding color models: A review, ARPN Journal of Science and Technology, 2 (2012), 265-275.   Google Scholar [7] Z. Jia, M. K. Ng and W. Wang, Color image restoration by saturation-value (SV) total variation, SIAM J. Imaging Sci., 12 (2019), 972-1000.  doi: 10.1137/18M1230451.  Google Scholar [8] S. B. Kang, Automatic removal of chromatic aberration from a single image, in 2007 IEEE Conference on Computer Vision and Pattern Recognition, 2007, 1–8. Google Scholar [9] H. Kang, S.-H. Lee, J. Chang and M. G. Kang, Partial differential equation-based approach for removal of chromatic aberration with local characteristics, J. of Electronic Imaging, 19 (2010), 1-8.   Google Scholar [10] V. Kaufmann and R. Ladstädter, Elimination of color fringes in digital photographs caused by lateral chromatic aberration, in Proc. 20th Int. Comm. Int. Photogramm. Archit. Symp. Conf., Oct., 2005, 1–6. Google Scholar [11] H. K. Kelda and P. Kaur, A review: Color models in image processing, Int. J. Computer Technology and Applications, 5 (2014), 319-322.   Google Scholar [12] B. Kim and R. Park, Automatic detection and correction of purple fringing using the gradient information and desaturation, in Proc. 16th Eur. Signal Process. Conf., Oct., 2008, 1–5. Google Scholar [13] R. Kimmel, Demosaicing: Image reconstruction from color CCD samples, IEEE Trans. Image Process., 8 (1999), 1221-1228.   Google Scholar [14] H. Kour, Analysis on image color model, International Journal of Advanced Research in Computer and Communication Engineering, 4 (2015), 1-3.   Google Scholar [15] P. B. Kruger, S. Mathews, K. R. Aggarwala and N. Sanchez, Chromatic aberration and ocular focus: Fincham revisited, Vision Research, 33 (1993), 1397-1411.  doi: 10.1016/0042-6989(93)90046-Y.  Google Scholar [16] J. Mallon and P. F. Whelan, Calibration and removal of lateral chromatic aberration in images, Pattern Recognition Letters, 28 (2007), 125-135.  doi: 10.1016/j.patrec.2006.06.013.  Google Scholar [17] D. H. Marimont and B. A. Wandell, Matching color images: The effects of axial chromatic aberration, Journal of the Optical Society of America A, 11 (1994), 3113-3122.  doi: 10.1364/JOSAA.11.003113.  Google Scholar [18] D. Martin, C. Fowlkes, D. Tal and J. Malik, A Database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics, Proceedings of IEEE International Conference on Computer Vision (ICCV) 2001, 2 (2001), 416-423.  doi: 10.1109/ICCV.2001.937655.  Google Scholar [19] P. Mouroulis and J. Macdonald, Geometrical Optics and Optical Design, 1$^{st}$ edition, Oxford U. Press, New York, 1997.   Google Scholar [20] L. Pi, W. Wang and M. Ng, A spatially variant total variational model for chromatic aberration correction, Journal of Visual Communication and Image Representation, 41 (2016), 296-304.  doi: 10.1016/j.jvcir.2016.10.009.  Google Scholar [21] L. N. Thibos, A. Bradley, D. L. Still, X. Zhang and P. A. Howarth, Theory and measurement of ocular chromatic aberration, Vision Research, 30 (1990), 33-49.  doi: 10.1016/0042-6989(90)90126-6.  Google Scholar [22] R. G. Willson and S. A. Shafer, Active lens control for high precision computer imaging, IEEE International Conference on Robotics and Automation, 3 (1991), 2063-2070.  doi: 10.1109/ROBOT.1991.131931.  Google Scholar [23] X. Zhang and B. A. Wandell, A spatial extension of CIELAB for digital color-image reproduction, Journal of the Society for Information Display, 5 (1997), 61-63.  doi: 10.1889/1.1985127.  Google Scholar
First row: The ground-truth image; Second row: the CA image; Third row: the CA correction image by using the local PDE method [9]; Forth row: the CA correction image by using the TV method [20] with $\mu = 45$; Fifth row: the CA corrected image by using the proposed method with $\mu = 20$. Note that the intensity values of red, green and blue colors are shown in line profile images
First row: The ground-truth image; Second row: the CA image; Third row: the CA correction image by using the local PDE method [9]; Forth row: the CA correction image by using the TV method $\mu = 45$ [20]; Fifth row: the CA corrected image by using the proposed method $\mu = 20$. Note that the intensity values of red, green and blue colors are shown in line profile images
First row: CA image, Ground-truth, zooming parts; First column (except first row): the restored results by using TV-CAC model with $\mu = 5, 10, 15, 20, 25$; Second column (except first row): the restored results by using SVTV-CAC model with $\mu = 5, 10, 15, 20, 25$; Third column (except first row): zooming parts of the restored results by using TV-CAC model; Forth column (except first row): zooming parts of the restored results by using SVTV-CAC model
Measure distribution
From left to right: ground-truth image, CA image, the restored results by using PDE model, the restored results by using TV-CAC model $\mu = 1$, the restored results by using SVTV-CAC model $\mu = 20$. Second row: the corresponding zooming parts. Third row: S-CIELAB color error distribution
From left to right: ground-truth image, CA image, the restored results by using PDE model, the restored results by using TV-CAC model $\mu = 1$, the restored results by using SVTV-CAC model $\mu = 10$. Second row: the corresponding zooming parts. Third row: S-CIELAB color error distribution
From left to right: ground-truth image, CA image, the restored results by using PDE model, the restored results by using TV-CAC model $\mu = 5$, the restored results by using SVTV-CAC model $\mu = 15$. Second row: the corresponding zooming parts. Third row: S-CIELAB color error distribution
From left to right: ground-truth image, CA image, the restored results by using PDE model, the restored results by using TV-CAC model $\mu = 0.5$, the restored results by using SVTV-CAC model $\mu = 15$. Second row: the corresponding zooming parts. Third row: S-CIELAB color error distribution
From left to right: ground-truth image, CA image, the restored results by using PDE model, the restored results by using TV-CAC model $\mu = 1$, the restored results by using SVTV-CAC model $\mu = 20$. Second row: the corresponding zooming parts. Third row: S-CIELAB color error distribution
First row: ground-truth image and CA image; Second row: the restored results by using GW and LC1 respectively; Third row: the restored results by using LC2, FCF and SVTV-CAC respectively
First row: original real world image and zooming part; Second row: the restored results by using GW and LC1 respectively; Third row: the restored results by using LC2, and FCF respectively; Fourth row: the restored result by using SVTV-CAC
Measure values with respect to $\mu$
 Figure method PSNR SSIM SCIELAB Error 1 by TV-CAC $\mu = 30$ 26.8443 0.9745 76557 $\mu = 35$ 27.5209 0.9763 69818 $\mu = 40$ 28.3432 0.9784 60924 $\mu = 45$ 28.8332 0.9796 55742 $\mu = 50$ 28.6660 0.9793 56195 $\mu = 55$ 27.9948 0.9778 58992 1 by SVTV-CAC $\mu = 10$ 28.9121 0.9739 62781 $\mu = 15$ 30.6855 0.9793 44484 $\mu = 20$ 32.1152 0.9824 33414 $\mu = 25$ 30.4036 0.9809 38666 $\mu = 30$ 28.7030 0.9774 47549 $\mu = 35$ 27.8225 0.9744 56347 2 by TV-CAC $\mu = 25$ 26.3643 0.9764 132157 $\mu = 30$ 26.4490 0.9764 129911 $\mu = 35$ 26.5936 0.9766 125456 $\mu = 40$ 26.7426 0.9765 120319 $\mu = 45$ 26.8504 0.9761 115719 $\mu = 50$ 26.8937 0.9750 111624 $\mu = 55$ 26.8625 0.9735 108241 $\mu = 60$ 26.7493 0.9712 104724 $\mu = 65$ 26.5678 0.9685 102599 2 by SVTV-CAC $\mu = 10$ 27.5949 0.9683 39666 $\mu = 15$ 28.0019 0.9724 33962 $\mu = 20$ 28.4454 0.9743 34005 $\mu = 25$ 28.3332 0.9738 35153 $\mu = 30$ 27.5742 0.9693 40700 $\mu = 35$ 26.4950 0.9604 47721
 Figure method PSNR SSIM SCIELAB Error 1 by TV-CAC $\mu = 30$ 26.8443 0.9745 76557 $\mu = 35$ 27.5209 0.9763 69818 $\mu = 40$ 28.3432 0.9784 60924 $\mu = 45$ 28.8332 0.9796 55742 $\mu = 50$ 28.6660 0.9793 56195 $\mu = 55$ 27.9948 0.9778 58992 1 by SVTV-CAC $\mu = 10$ 28.9121 0.9739 62781 $\mu = 15$ 30.6855 0.9793 44484 $\mu = 20$ 32.1152 0.9824 33414 $\mu = 25$ 30.4036 0.9809 38666 $\mu = 30$ 28.7030 0.9774 47549 $\mu = 35$ 27.8225 0.9744 56347 2 by TV-CAC $\mu = 25$ 26.3643 0.9764 132157 $\mu = 30$ 26.4490 0.9764 129911 $\mu = 35$ 26.5936 0.9766 125456 $\mu = 40$ 26.7426 0.9765 120319 $\mu = 45$ 26.8504 0.9761 115719 $\mu = 50$ 26.8937 0.9750 111624 $\mu = 55$ 26.8625 0.9735 108241 $\mu = 60$ 26.7493 0.9712 104724 $\mu = 65$ 26.5678 0.9685 102599 2 by SVTV-CAC $\mu = 10$ 27.5949 0.9683 39666 $\mu = 15$ 28.0019 0.9724 33962 $\mu = 20$ 28.4454 0.9743 34005 $\mu = 25$ 28.3332 0.9738 35153 $\mu = 30$ 27.5742 0.9693 40700 $\mu = 35$ 26.4950 0.9604 47721
Measure values of different models
 Figure method PSNR SSIM SCIELAB Error 5 PDE 21.0142 0.9076 52054 TV 22.9057 0.9587 54186 SVTV 26.1505 0.9644 32019 6 PDE 27.2024 0.9531 24001 TV 26.6299 0.9603 50560 SVTV 27.8191 0.9611 21604 7 PDE 25.5506 0.9533 29722 TV 27.4673 0.9762 39483 SVTV 28.2084 0.9776 18815 8 PDE 24.9437 0.9389 27986 TV 27.4506 0.9787 65008 SVTV 30.0501 0.9835 9287 9 PDE 35.9356 0.9864 6197 TV 35.4925 0.9883 6444 SVTV 36.3883 0.9892 4130
 Figure method PSNR SSIM SCIELAB Error 5 PDE 21.0142 0.9076 52054 TV 22.9057 0.9587 54186 SVTV 26.1505 0.9644 32019 6 PDE 27.2024 0.9531 24001 TV 26.6299 0.9603 50560 SVTV 27.8191 0.9611 21604 7 PDE 25.5506 0.9533 29722 TV 27.4673 0.9762 39483 SVTV 28.2084 0.9776 18815 8 PDE 24.9437 0.9389 27986 TV 27.4506 0.9787 65008 SVTV 30.0501 0.9835 9287 9 PDE 35.9356 0.9864 6197 TV 35.4925 0.9883 6444 SVTV 36.3883 0.9892 4130
Measure values of different models
 Figure method PSNR SSIM SCIELAB Error 10 CM1 26.2709 0.8654 18085 CM2 24.9585 0.8485 23807 CM3 28.2337 0.8956 13936 PM 26.3282 0.8559 15946 SVTV 31.5037 0.9450 6768
 Figure method PSNR SSIM SCIELAB Error 10 CM1 26.2709 0.8654 18085 CM2 24.9585 0.8485 23807 CM3 28.2337 0.8956 13936 PM 26.3282 0.8559 15946 SVTV 31.5037 0.9450 6768
 [1] Yuyuan Ouyang, Yunmei Chen, Ying Wu. Total variation and wavelet regularization of orientation distribution functions in diffusion MRI. Inverse Problems & Imaging, 2013, 7 (2) : 565-583. doi: 10.3934/ipi.2013.7.565 [2] Juan Carlos De los Reyes, Estefanía Loayza-Romero. Total generalized variation regularization in data assimilation for Burgers' equation. Inverse Problems & Imaging, 2019, 13 (4) : 755-786. doi: 10.3934/ipi.2019035 [3] J. Mead. $\chi^2$ test for total variation regularization parameter selection. Inverse Problems & Imaging, 2020, 14 (3) : 401-421. doi: 10.3934/ipi.2020019 [4] You-Wei Wen, Raymond Honfu Chan. Using generalized cross validation to select regularization parameter for total variation regularization problems. Inverse Problems & Imaging, 2018, 12 (5) : 1103-1120. doi: 10.3934/ipi.2018046 [5] Raymond H. Chan, Haixia Liang, Suhua Wei, Mila Nikolova, Xue-Cheng Tai. High-order total variation regularization approach for axially symmetric object tomography from a single radiograph. Inverse Problems & Imaging, 2015, 9 (1) : 55-77. doi: 10.3934/ipi.2015.9.55 [6] Xiaoqun Zhang, Tony F. Chan. Wavelet inpainting by nonlocal total variation. Inverse Problems & Imaging, 2010, 4 (1) : 191-210. doi: 10.3934/ipi.2010.4.191 [7] Rinaldo M. Colombo, Francesca Monti. Solutions with large total variation to nonconservative hyperbolic systems. Communications on Pure & Applied Analysis, 2010, 9 (1) : 47-60. doi: 10.3934/cpaa.2010.9.47 [8] Ryan Compton, Stanley Osher, Louis-S. Bouchard. Hybrid regularization for MRI reconstruction with static field inhomogeneity correction. Inverse Problems & Imaging, 2013, 7 (4) : 1215-1233. doi: 10.3934/ipi.2013.7.1215 [9] Yunho Kim, Paul M. Thompson, Luminita A. Vese. HARDI data denoising using vectorial total variation and logarithmic barrier. Inverse Problems & Imaging, 2010, 4 (2) : 273-310. doi: 10.3934/ipi.2010.4.273 [10] Sören Bartels, Marijo Milicevic. Iterative finite element solution of a constrained total variation regularized model problem. Discrete & Continuous Dynamical Systems - S, 2017, 10 (6) : 1207-1232. doi: 10.3934/dcdss.2017066 [11] Yunhai Xiao, Junfeng Yang, Xiaoming Yuan. Alternating algorithms for total variation image reconstruction from random projections. Inverse Problems & Imaging, 2012, 6 (3) : 547-563. doi: 10.3934/ipi.2012.6.547 [12] Juan C. Moreno, V. B. Surya Prasath, João C. Neves. Color image processing by vectorial total variation with gradient channels coupling. Inverse Problems & Imaging, 2016, 10 (2) : 461-497. doi: 10.3934/ipi.2016008 [13] Liyan Ma, Lionel Moisan, Jian Yu, Tieyong Zeng. A stable method solving the total variation dictionary model with $L^\infty$ constraints. Inverse Problems & Imaging, 2014, 8 (2) : 507-535. doi: 10.3934/ipi.2014.8.507 [14] Lu Liu, Zhi-Feng Pang, Yuping Duan. Retinex based on exponent-type total variation scheme. Inverse Problems & Imaging, 2018, 12 (5) : 1199-1217. doi: 10.3934/ipi.2018050 [15] Chunlin Wu, Juyong Zhang, Xue-Cheng Tai. Augmented Lagrangian method for total variation restoration with non-quadratic fidelity. Inverse Problems & Imaging, 2011, 5 (1) : 237-261. doi: 10.3934/ipi.2011.5.237 [16] Florian Krügel. Some properties of minimizers of a variational problem involving the total variation functional. Communications on Pure & Applied Analysis, 2015, 14 (1) : 341-360. doi: 10.3934/cpaa.2015.14.341 [17] Zhengmeng Jin, Chen Zhou, Michael K. Ng. A coupled total variation model with curvature driven for image colorization. Inverse Problems & Imaging, 2016, 10 (4) : 1037-1055. doi: 10.3934/ipi.2016031 [18] Adriana González, Laurent Jacques, Christophe De Vleeschouwer, Philippe Antoine. Compressive optical deflectometric tomography: A constrained total-variation minimization approach. Inverse Problems & Imaging, 2014, 8 (2) : 421-457. doi: 10.3934/ipi.2014.8.421 [19] Johnathan M. Bardsley. An efficient computational method for total variation-penalized Poisson likelihood estimation. Inverse Problems & Imaging, 2008, 2 (2) : 167-185. doi: 10.3934/ipi.2008.2.167 [20] Konstantinos Papafitsoros, Kristian Bredies. A study of the one dimensional total generalised variation regularisation problem. Inverse Problems & Imaging, 2015, 9 (2) : 511-550. doi: 10.3934/ipi.2015.9.511

2018 Impact Factor: 1.469

## Tools

Article outline

Figures and Tables