A coupled total variation model with curvature driven for image colorization

Zhengmeng  Jin; Chen  Zhou; Michael K. Ng

doi:10.3934/ipi.2016031

Article Contents

2016, Volume 10, Issue 4: 1037-1055. Doi: 10.3934/ipi.2016031

This issue Previous Article The Bayesian formulation of EIT: Analysis and algorithms Next Article A globally convergent numerical method for a 1-d inverse medium problem with experimental data

A coupled total variation model with curvature driven for image colorization

1.
School of Science, Nanjing University of Posts and Telecommunications, Nanjing
2.
Centre for Mathematical Imaging and Vision and Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong

Received: September 2015

Revised: December 2015

Published: November 2016

Abstract / Introduction Related Papers Cited by

Abstract

In this paper, we study the problem of image colorization based on the propagation from given color pixels to the other grey-level pixels in grayscale images. We propose to use a coupled total variation model with curvature information of luminance channel to control the colorization process. There are two distinct advantages of the proposed model: (i) the involved optimization problem is convex and it is not sensitive to initial guess of colorization procedure; (ii) the proposed model makes use of curvature information to control the color diffusion process which is more effectively than that by using the gradient information. The existence of the minimizer of the proposed model can be shown, and the numerical solver based on convex programming techniques can be developed to solve the resulting model very efficiently. Experimental results are reported to demonstrate that the performance of the proposed model is better than those of the other color propagation models, especially when we deal with large regions of grayscale images for colorization.

Keywords:

Colorization,
coupled total variation,
curvature,
convexity,
alternating direction method of multipliers.

Mathematics Subject Classification: 68U10, 49J40, 65K10.

Citation:

Related Papers

Cited by

References

[1]	L. Ambrosio, N. Fusco and D. Pallara, Functions of Bounded Variation and Free Discontinuity Problems, Oxford University Press, 2000.
[2]	X. Bresson and T. F. Chan, Fast dual minimization of the vectorical total variation norm and application to color image processing, Inverse Problems and Imaging, 2 (2008), 455-484.doi: 10.3934/ipi.2008.2.455.
[3]	A. Bugeau, V.-T. Ta and N. Papadakis, Variational exemplar-based image colorization, IEEE Trans. Image Process., 23 (2014), 298-307.doi: 10.1109/TIP.2013.2288929.
[4]	V. Caselles, G. Facciolo and E. Meinhardt, Anisotropic Cheeger sets and applications, SIAM J. Imaging Sci., 2 (2009), 1211-1254.doi: 10.1137/08073696X.
[5]	R. H. Chan, J. Yang and X. Yuan, Alternating Direction Method for Image Inpainting in Wavelet Domains, SIAM J. Imaging Sci., 4 (2011), 807-826.doi: 10.1137/100807247.
[6]	Y. Chen and T. Wunderli, Adaptive total variation for image restoration in BV space, J. Math. Anal. Appl., 272 (2002), 117-137.doi: 10.1016/S0022-247X(02)00141-5.
[7]	Z. M. Jin, F. Li and M. K. Ng, A variational approach for image decolorization by variance maximization, SIAM J. Imaging Sci., 7 (2014), 944-968.doi: 10.1137/130935197.
[8]	S. H. Kang and R. March, Variational models for image colorization via chromaticity and brightness decomposition, IEEE Transactions On Image Processing, 16 (2007), 2251-2261.doi: 10.1109/TIP.2007.903257.
[9]	A. Levin, D. Lischinski and Y. Weiss, Colorization using optimization, in Proc. SIGGRAPH Conf., 23 (2004), 689-694.doi: 10.1145/1186562.1015780.
[10]	F. Pierre, J.-F. Aujol, A. Bugeau, N. Papadakis and V.-T. Ta, Luminance-chrominance model for image colorization, SIAM J. Imaging Sci., 8 (2015), 536-563.doi: 10.1137/140979368.
[11]	M. H. Quang, S. H. Kang and T. M. Le, Image and video colorization using vector-valued reproducing kernel Hilbert spaces, Journal of Mathematical Imaging and Vision, 37 (2010), 49-65.doi: 10.1007/s10851-010-0192-8.
[12]	M. L. Song, D. C. Tao and C. Chen, Color to gray: Visual cue preservation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 32 (2012), 1537-1552.
[13]	J. Eckstein and D. Bertsekas, On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators, Math. Programming, 55 (1992), 293-318.doi: 10.1007/BF01581204.
[14]	D. Gabay and B. Mercier, A dual algorithm for the solution of nonlinear variational problems via finite-element approximations, Comp. Math. Appl., 2 (1976), 17-40.doi: 10.1016/0898-1221(76)90003-1.
[15]	T. Goldstein and S. Osher, The split Bregman method for $L^1$ regularized problems, SIAM Journal on Imaging Sciences, 2 (2009), 323-343.doi: 10.1137/080725891.
[16]	X. D. Hou, J. Harel and C. Koch, Image signature: Highlighting sparse salient regions, IEEE Trans. Pattern Anal. Mach. Intell., 34 (2012), 194-201.
[17]	R. Hunter, Photoelectric color difference meter, Journal of the Optical Society of America, 48 (2008), 985-993.doi: 10.1364/JOSA.48.000985.
[18]	R. Irony, D. Cohen and D. Lischinski, Colorization by example, in Eurographics conference on Rendering Techniques, Eurographics Association, (2005), 201-210.
[19]	W. K. Pratt, Ed., Digital Image Processing, Wilely-Interscience publication, New York, 2001.
[20]	K. Ito and K. Kunisch, An augmented Lagrangian technique for variational inequalities, Appl. Math. Optim., 21 (1990), 223-241.doi: 10.1007/BF01445164.
[21]	L. Itti, C. Koch and E. Niebur, A model of saliency-based visual attention for rapid scene analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, 20 (1998), 1254-1259.doi: 10.1109/34.730558.
[22]	M. K. Ng, P. Weiss and X. Yuan, Solving constrained total-variation image restoration and reconstruction problems via alternating direction methods, SIAM J. Sci. Comput., 32 (2010), 2710-2736.doi: 10.1137/090774823.
[23]	F. Pierre, J.-F. Aujol, A. Bugeau, N. Papadakis and V.-T. Ta, Luminance-chrominance model for image colorization, SIAM J. Imaging Sciences, 8 (2015), 536-563.doi: 10.1137/140979368.
[24]	F. Pierre, J.-F. Aujol, A. Bugeau, N. Papadakis and V.-T. Ta, A unified model for image colorization, computer vision - ECCV 2014 workshops, Lecture Notes in Computer Science, 8927 (2015), 297-308.
[25]	G. Sapiro, Inpainting the colors, in Proc. IEEE Int. Conf. Image Processing, 2 (2005), 698-701.doi: 10.1109/ICIP.2005.1530151.
[26]	G. Sapiro, Geometric Partial Differential Equations and Image Analysis, Cambridge university press, 2006.
[27]	A. A. Shah, M. Gandhi and K. M. Shah, Medical image colorization uisng optimization technique, International Journal of Scientific and Research Publications, 3 (2013), 1-6.
[28]	T. Welsh, M. Ashikhmin and K. Mueller, Transferring color to grayscale images, ACM Transactions on Graphics, 21 (2002), 277-280.
[29]	L. Yatziv and G. Sapiro, Fast image and video colorization using chrominance blending, IEEE Trans. Image Process., 15 (2006), 1120-1129.doi: 10.1109/TIP.2005.864231.