November  2016, 10(4): 1037-1055. doi: 10.3934/ipi.2016031

A coupled total variation model with curvature driven for image colorization

1. 

School of Science, Nanjing University of Posts and Telecommunications, Nanjing, China, China

2. 

Centre for Mathematical Imaging and Vision and Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong

Received  September 2015 Revised  December 2015 Published  October 2016

In this paper, we study the problem of image colorization based on the propagation from given color pixels to the other grey-level pixels in grayscale images. We propose to use a coupled total variation model with curvature information of luminance channel to control the colorization process. There are two distinct advantages of the proposed model: (i) the involved optimization problem is convex and it is not sensitive to initial guess of colorization procedure; (ii) the proposed model makes use of curvature information to control the color diffusion process which is more effectively than that by using the gradient information. The existence of the minimizer of the proposed model can be shown, and the numerical solver based on convex programming techniques can be developed to solve the resulting model very efficiently. Experimental results are reported to demonstrate that the performance of the proposed model is better than those of the other color propagation models, especially when we deal with large regions of grayscale images for colorization.
Citation: Zhengmeng Jin, Chen Zhou, Michael K. Ng. A coupled total variation model with curvature driven for image colorization. Inverse Problems & Imaging, 2016, 10 (4) : 1037-1055. doi: 10.3934/ipi.2016031
References:
[1]

L. Ambrosio, N. Fusco and D. Pallara, Functions of Bounded Variation and Free Discontinuity Problems,, Oxford University Press, (2000).   Google Scholar

[2]

X. Bresson and T. F. Chan, Fast dual minimization of the vectorical total variation norm and application to color image processing,, Inverse Problems and Imaging, 2 (2008), 455.  doi: 10.3934/ipi.2008.2.455.  Google Scholar

[3]

A. Bugeau, V.-T. Ta and N. Papadakis, Variational exemplar-based image colorization,, IEEE Trans. Image Process., 23 (2014), 298.  doi: 10.1109/TIP.2013.2288929.  Google Scholar

[4]

V. Caselles, G. Facciolo and E. Meinhardt, Anisotropic Cheeger sets and applications,, SIAM J. Imaging Sci., 2 (2009), 1211.  doi: 10.1137/08073696X.  Google Scholar

[5]

R. H. Chan, J. Yang and X. Yuan, Alternating Direction Method for Image Inpainting in Wavelet Domains,, SIAM J. Imaging Sci., 4 (2011), 807.  doi: 10.1137/100807247.  Google Scholar

[6]

Y. Chen and T. Wunderli, Adaptive total variation for image restoration in BV space,, J. Math. Anal. Appl., 272 (2002), 117.  doi: 10.1016/S0022-247X(02)00141-5.  Google Scholar

[7]

Z. M. Jin, F. Li and M. K. Ng, A variational approach for image decolorization by variance maximization,, SIAM J. Imaging Sci., 7 (2014), 944.  doi: 10.1137/130935197.  Google Scholar

[8]

S. H. Kang and R. March, Variational models for image colorization via chromaticity and brightness decomposition,, IEEE Transactions On Image Processing, 16 (2007), 2251.  doi: 10.1109/TIP.2007.903257.  Google Scholar

[9]

A. Levin, D. Lischinski and Y. Weiss, Colorization using optimization,, in Proc. SIGGRAPH Conf., 23 (2004), 689.  doi: 10.1145/1186562.1015780.  Google Scholar

[10]

F. Pierre, J.-F. Aujol, A. Bugeau, N. Papadakis and V.-T. Ta, Luminance-chrominance model for image colorization,, SIAM J. Imaging Sci., 8 (2015), 536.  doi: 10.1137/140979368.  Google Scholar

[11]

M. H. Quang, S. H. Kang and T. M. Le, Image and video colorization using vector-valued reproducing kernel Hilbert spaces,, Journal of Mathematical Imaging and Vision, 37 (2010), 49.  doi: 10.1007/s10851-010-0192-8.  Google Scholar

[12]

M. L. Song, D. C. Tao and C. Chen, Color to gray: Visual cue preservation,, IEEE Transactions on Pattern Analysis and Machine Intelligence, 32 (2012), 1537.   Google Scholar

[13]

J. Eckstein and D. Bertsekas, On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators,, Math. Programming, 55 (1992), 293.  doi: 10.1007/BF01581204.  Google Scholar

[14]

D. Gabay and B. Mercier, A dual algorithm for the solution of nonlinear variational problems via finite-element approximations,, Comp. Math. Appl., 2 (1976), 17.  doi: 10.1016/0898-1221(76)90003-1.  Google Scholar

[15]

T. Goldstein and S. Osher, The split Bregman method for $L^1$ regularized problems,, SIAM Journal on Imaging Sciences, 2 (2009), 323.  doi: 10.1137/080725891.  Google Scholar

[16]

X. D. Hou, J. Harel and C. Koch, Image signature: Highlighting sparse salient regions,, IEEE Trans. Pattern Anal. Mach. Intell., 34 (2012), 194.   Google Scholar

[17]

R. Hunter, Photoelectric color difference meter,, Journal of the Optical Society of America, 48 (2008), 985.  doi: 10.1364/JOSA.48.000985.  Google Scholar

[18]

R. Irony, D. Cohen and D. Lischinski, Colorization by example, in Eurographics conference on Rendering Techniques,, Eurographics Association, (2005), 201.   Google Scholar

[19]

W. K. Pratt, Ed., Digital Image Processing, Wilely-Interscience publication,, New York, (2001).   Google Scholar

[20]

K. Ito and K. Kunisch, An augmented Lagrangian technique for variational inequalities,, Appl. Math. Optim., 21 (1990), 223.  doi: 10.1007/BF01445164.  Google Scholar

[21]

L. Itti, C. Koch and E. Niebur, A model of saliency-based visual attention for rapid scene analysis,, IEEE Transactions on Pattern Analysis and Machine Intelligence, 20 (1998), 1254.  doi: 10.1109/34.730558.  Google Scholar

[22]

M. K. Ng, P. Weiss and X. Yuan, Solving constrained total-variation image restoration and reconstruction problems via alternating direction methods,, SIAM J. Sci. Comput., 32 (2010), 2710.  doi: 10.1137/090774823.  Google Scholar

[23]

F. Pierre, J.-F. Aujol, A. Bugeau, N. Papadakis and V.-T. Ta, Luminance-chrominance model for image colorization,, SIAM J. Imaging Sciences, 8 (2015), 536.  doi: 10.1137/140979368.  Google Scholar

[24]

F. Pierre, J.-F. Aujol, A. Bugeau, N. Papadakis and V.-T. Ta, A unified model for image colorization, computer vision - ECCV 2014 workshops,, Lecture Notes in Computer Science, 8927 (2015), 297.   Google Scholar

[25]

G. Sapiro, Inpainting the colors,, in Proc. IEEE Int. Conf. Image Processing, 2 (2005), 698.  doi: 10.1109/ICIP.2005.1530151.  Google Scholar

[26]

G. Sapiro, Geometric Partial Differential Equations and Image Analysis,, Cambridge university press, (2006).   Google Scholar

[27]

A. A. Shah, M. Gandhi and K. M. Shah, Medical image colorization uisng optimization technique,, International Journal of Scientific and Research Publications, 3 (2013), 1.   Google Scholar

[28]

T. Welsh, M. Ashikhmin and K. Mueller, Transferring color to grayscale images,, ACM Transactions on Graphics, 21 (2002), 277.   Google Scholar

[29]

L. Yatziv and G. Sapiro, Fast image and video colorization using chrominance blending,, IEEE Trans. Image Process., 15 (2006), 1120.  doi: 10.1109/TIP.2005.864231.  Google Scholar

show all references

References:
[1]

L. Ambrosio, N. Fusco and D. Pallara, Functions of Bounded Variation and Free Discontinuity Problems,, Oxford University Press, (2000).   Google Scholar

[2]

X. Bresson and T. F. Chan, Fast dual minimization of the vectorical total variation norm and application to color image processing,, Inverse Problems and Imaging, 2 (2008), 455.  doi: 10.3934/ipi.2008.2.455.  Google Scholar

[3]

A. Bugeau, V.-T. Ta and N. Papadakis, Variational exemplar-based image colorization,, IEEE Trans. Image Process., 23 (2014), 298.  doi: 10.1109/TIP.2013.2288929.  Google Scholar

[4]

V. Caselles, G. Facciolo and E. Meinhardt, Anisotropic Cheeger sets and applications,, SIAM J. Imaging Sci., 2 (2009), 1211.  doi: 10.1137/08073696X.  Google Scholar

[5]

R. H. Chan, J. Yang and X. Yuan, Alternating Direction Method for Image Inpainting in Wavelet Domains,, SIAM J. Imaging Sci., 4 (2011), 807.  doi: 10.1137/100807247.  Google Scholar

[6]

Y. Chen and T. Wunderli, Adaptive total variation for image restoration in BV space,, J. Math. Anal. Appl., 272 (2002), 117.  doi: 10.1016/S0022-247X(02)00141-5.  Google Scholar

[7]

Z. M. Jin, F. Li and M. K. Ng, A variational approach for image decolorization by variance maximization,, SIAM J. Imaging Sci., 7 (2014), 944.  doi: 10.1137/130935197.  Google Scholar

[8]

S. H. Kang and R. March, Variational models for image colorization via chromaticity and brightness decomposition,, IEEE Transactions On Image Processing, 16 (2007), 2251.  doi: 10.1109/TIP.2007.903257.  Google Scholar

[9]

A. Levin, D. Lischinski and Y. Weiss, Colorization using optimization,, in Proc. SIGGRAPH Conf., 23 (2004), 689.  doi: 10.1145/1186562.1015780.  Google Scholar

[10]

F. Pierre, J.-F. Aujol, A. Bugeau, N. Papadakis and V.-T. Ta, Luminance-chrominance model for image colorization,, SIAM J. Imaging Sci., 8 (2015), 536.  doi: 10.1137/140979368.  Google Scholar

[11]

M. H. Quang, S. H. Kang and T. M. Le, Image and video colorization using vector-valued reproducing kernel Hilbert spaces,, Journal of Mathematical Imaging and Vision, 37 (2010), 49.  doi: 10.1007/s10851-010-0192-8.  Google Scholar

[12]

M. L. Song, D. C. Tao and C. Chen, Color to gray: Visual cue preservation,, IEEE Transactions on Pattern Analysis and Machine Intelligence, 32 (2012), 1537.   Google Scholar

[13]

J. Eckstein and D. Bertsekas, On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators,, Math. Programming, 55 (1992), 293.  doi: 10.1007/BF01581204.  Google Scholar

[14]

D. Gabay and B. Mercier, A dual algorithm for the solution of nonlinear variational problems via finite-element approximations,, Comp. Math. Appl., 2 (1976), 17.  doi: 10.1016/0898-1221(76)90003-1.  Google Scholar

[15]

T. Goldstein and S. Osher, The split Bregman method for $L^1$ regularized problems,, SIAM Journal on Imaging Sciences, 2 (2009), 323.  doi: 10.1137/080725891.  Google Scholar

[16]

X. D. Hou, J. Harel and C. Koch, Image signature: Highlighting sparse salient regions,, IEEE Trans. Pattern Anal. Mach. Intell., 34 (2012), 194.   Google Scholar

[17]

R. Hunter, Photoelectric color difference meter,, Journal of the Optical Society of America, 48 (2008), 985.  doi: 10.1364/JOSA.48.000985.  Google Scholar

[18]

R. Irony, D. Cohen and D. Lischinski, Colorization by example, in Eurographics conference on Rendering Techniques,, Eurographics Association, (2005), 201.   Google Scholar

[19]

W. K. Pratt, Ed., Digital Image Processing, Wilely-Interscience publication,, New York, (2001).   Google Scholar

[20]

K. Ito and K. Kunisch, An augmented Lagrangian technique for variational inequalities,, Appl. Math. Optim., 21 (1990), 223.  doi: 10.1007/BF01445164.  Google Scholar

[21]

L. Itti, C. Koch and E. Niebur, A model of saliency-based visual attention for rapid scene analysis,, IEEE Transactions on Pattern Analysis and Machine Intelligence, 20 (1998), 1254.  doi: 10.1109/34.730558.  Google Scholar

[22]

M. K. Ng, P. Weiss and X. Yuan, Solving constrained total-variation image restoration and reconstruction problems via alternating direction methods,, SIAM J. Sci. Comput., 32 (2010), 2710.  doi: 10.1137/090774823.  Google Scholar

[23]

F. Pierre, J.-F. Aujol, A. Bugeau, N. Papadakis and V.-T. Ta, Luminance-chrominance model for image colorization,, SIAM J. Imaging Sciences, 8 (2015), 536.  doi: 10.1137/140979368.  Google Scholar

[24]

F. Pierre, J.-F. Aujol, A. Bugeau, N. Papadakis and V.-T. Ta, A unified model for image colorization, computer vision - ECCV 2014 workshops,, Lecture Notes in Computer Science, 8927 (2015), 297.   Google Scholar

[25]

G. Sapiro, Inpainting the colors,, in Proc. IEEE Int. Conf. Image Processing, 2 (2005), 698.  doi: 10.1109/ICIP.2005.1530151.  Google Scholar

[26]

G. Sapiro, Geometric Partial Differential Equations and Image Analysis,, Cambridge university press, (2006).   Google Scholar

[27]

A. A. Shah, M. Gandhi and K. M. Shah, Medical image colorization uisng optimization technique,, International Journal of Scientific and Research Publications, 3 (2013), 1.   Google Scholar

[28]

T. Welsh, M. Ashikhmin and K. Mueller, Transferring color to grayscale images,, ACM Transactions on Graphics, 21 (2002), 277.   Google Scholar

[29]

L. Yatziv and G. Sapiro, Fast image and video colorization using chrominance blending,, IEEE Trans. Image Process., 15 (2006), 1120.  doi: 10.1109/TIP.2005.864231.  Google Scholar

[1]

Yuan Shen, Lei Ji. Partial convolution for total variation deblurring and denoising by new linearized alternating direction method of multipliers with extension step. Journal of Industrial & Management Optimization, 2019, 15 (1) : 159-175. doi: 10.3934/jimo.2018037

[2]

Russell E. Warren, Stanley J. Osher. Hyperspectral unmixing by the alternating direction method of multipliers. Inverse Problems & Imaging, 2015, 9 (3) : 917-933. doi: 10.3934/ipi.2015.9.917

[3]

Sohana Jahan. Supervised distance preserving projection using alternating direction method of multipliers. Journal of Industrial & Management Optimization, 2017, 13 (5) : 1-17. doi: 10.3934/jimo.2019029

[4]

Foxiang Liu, Lingling Xu, Yuehong Sun, Deren Han. A proximal alternating direction method for multi-block coupled convex optimization. Journal of Industrial & Management Optimization, 2019, 15 (2) : 723-737. doi: 10.3934/jimo.2018067

[5]

Bingsheng He, Xiaoming Yuan. Linearized alternating direction method of multipliers with Gaussian back substitution for separable convex programming. Numerical Algebra, Control & Optimization, 2013, 3 (2) : 247-260. doi: 10.3934/naco.2013.3.247

[6]

Zhongming Wu, Xingju Cai, Deren Han. Linearized block-wise alternating direction method of multipliers for multiple-block convex programming. Journal of Industrial & Management Optimization, 2018, 14 (3) : 833-855. doi: 10.3934/jimo.2017078

[7]

Yunhai Xiao, Junfeng Yang, Xiaoming Yuan. Alternating algorithms for total variation image reconstruction from random projections. Inverse Problems & Imaging, 2012, 6 (3) : 547-563. doi: 10.3934/ipi.2012.6.547

[8]

Liyan Ma, Lionel Moisan, Jian Yu, Tieyong Zeng. A stable method solving the total variation dictionary model with $L^\infty$ constraints. Inverse Problems & Imaging, 2014, 8 (2) : 507-535. doi: 10.3934/ipi.2014.8.507

[9]

Chunlin Wu, Juyong Zhang, Xue-Cheng Tai. Augmented Lagrangian method for total variation restoration with non-quadratic fidelity. Inverse Problems & Imaging, 2011, 5 (1) : 237-261. doi: 10.3934/ipi.2011.5.237

[10]

Johnathan M. Bardsley. An efficient computational method for total variation-penalized Poisson likelihood estimation. Inverse Problems & Imaging, 2008, 2 (2) : 167-185. doi: 10.3934/ipi.2008.2.167

[11]

Yue Lu, Ying-En Ge, Li-Wei Zhang. An alternating direction method for solving a class of inverse semi-definite quadratic programming problems. Journal of Industrial & Management Optimization, 2016, 12 (1) : 317-336. doi: 10.3934/jimo.2016.12.317

[12]

Yunhai Xiao, Soon-Yi Wu, Bing-Sheng He. A proximal alternating direction method for $\ell_{2,1}$-norm least squares problem in multi-task feature learning. Journal of Industrial & Management Optimization, 2012, 8 (4) : 1057-1069. doi: 10.3934/jimo.2012.8.1057

[13]

Xiaoqun Zhang, Tony F. Chan. Wavelet inpainting by nonlocal total variation. Inverse Problems & Imaging, 2010, 4 (1) : 191-210. doi: 10.3934/ipi.2010.4.191

[14]

Rinaldo M. Colombo, Francesca Monti. Solutions with large total variation to nonconservative hyperbolic systems. Communications on Pure & Applied Analysis, 2010, 9 (1) : 47-60. doi: 10.3934/cpaa.2010.9.47

[15]

Yuan Shen, Xin Liu. An alternating minimization method for matrix completion problems. Discrete & Continuous Dynamical Systems - S, 2018, 0 (0) : 0-0. doi: 10.3934/dcdss.2020103

[16]

Yunho Kim, Paul M. Thompson, Luminita A. Vese. HARDI data denoising using vectorial total variation and logarithmic barrier. Inverse Problems & Imaging, 2010, 4 (2) : 273-310. doi: 10.3934/ipi.2010.4.273

[17]

Sören Bartels, Marijo Milicevic. Iterative finite element solution of a constrained total variation regularized model problem. Discrete & Continuous Dynamical Systems - S, 2017, 10 (6) : 1207-1232. doi: 10.3934/dcdss.2017066

[18]

Juan C. Moreno, V. B. Surya Prasath, João C. Neves. Color image processing by vectorial total variation with gradient channels coupling. Inverse Problems & Imaging, 2016, 10 (2) : 461-497. doi: 10.3934/ipi.2016008

[19]

Lu Liu, Zhi-Feng Pang, Yuping Duan. Retinex based on exponent-type total variation scheme. Inverse Problems & Imaging, 2018, 12 (5) : 1199-1217. doi: 10.3934/ipi.2018050

[20]

Florian Krügel. Some properties of minimizers of a variational problem involving the total variation functional. Communications on Pure & Applied Analysis, 2015, 14 (1) : 341-360. doi: 10.3934/cpaa.2015.14.341

2018 Impact Factor: 1.469

Metrics

  • PDF downloads (11)
  • HTML views (0)
  • Cited by (1)

Other articles
by authors

[Back to Top]