A colorization-based anisotropic variational model for vector-valued image compression

Yaqun Zhang; Wenjuan Yao; Zhichang Guo; Dazhi Zhang; Boying Wu

doi:10.3934/ipi.2022038

Article Contents

2023, Volume 17, Issue 1: 230-262. Doi: 10.3934/ipi.2022038

This issue Previous Article Data-consistent neural networks for solving nonlinear inverse problems Next Article Inverse transmission eigenvalue problem for fixed angular momentum

A colorization-based anisotropic variational model for vector-valued image compression

School of Mathematics, Harbin Institute of Technology, Harbin 150001, China

^*Corresponding author: Dazhi Zhang
^*Corresponding author: Dazhi Zhang

Received: December 2021

Revised: May 2022

Early access: July 27, 2022

Published online: July 27, 2022

Abstract / Introduction Full Text(HTML) Figure(11) / Table(5) Related Papers Cited by

Abstract

Image compression is an important technology in digital image processing. In this paper, a novel colorization-based codec for vector-valued images is proposed. In compression, we first define the concept of "structure image", which contains rich geometric structure information of the vector-valued image. Then, to extract representative pixels from the original vector-valued image, a "one-iteration method" is proposed. It can tremendously improve compression efficiency. In decompression, starting from colorizing the structure image, an anisotropic variational model is proposed. The existence and uniqueness of minimizers for the proposed variational model are established. Besides, we develop a fast and efficient algorithm for solving the model numerically by employing the scaled form of the alternating direction method of multipliers (ADMM). Numerical experiments on natural color images demonstrate that the proposed method outperforms the state-of-art colorization-based image compression method. Compared with the transform-based approaches, experiments on satellite multispectral images illustrate that the proposed method is superior to the JPEG and JPEG2000 standards.

Keywords:

Mathematics Subject Classification: Primary: 68U10, 94A08; Secondary: 32A70.

Citation:

Full Text(HTML)

Figure 1. $ \rm (a) $ Original images in the $ \rm RGB $ color space. $ \rm (b) $ Structure images obtained by taking $ (\alpha_1, \alpha_2, \alpha_3) = (0.299, 0.587, 0.114) $. $ \rm (c) $ Structure images obtained by taking a random set, $ \left(\alpha_1, \alpha_2, \alpha_3 \right) = (0.05, 0.3, 0.65) $. $ \rm (d) $ The difference between structure images (b) and (c). Note that the pixel values have been rearranged to [0, 255]

Download: Full-size image PowerPoint slide

Figure 2. $ \rm (a) $ When $ (\alpha_1, \alpha_2, \alpha_3) = (0.299, 0.587, 0.114) $, the geometric structure information in the structure image. $ \rm (b) $ The geometric structure information contained in the structure image, when $ \left(\alpha_1, \alpha_2, \alpha_3 \right) = (0.05, 0.3, 0.65) $. $ \rm (c) $ The difference of geometric structure information between (a) and (b). Pixel values have been rearranged into $ [0, 255] $ for better visibility

Download: Full-size image PowerPoint slide

Figure 3. Results of image change rate for Kodak images [19] $ (07, 23, 24) $ calculated by different methods

Download: Full-size image PowerPoint slide

Figure 4. Extracting representative pixels from the $ 96\times96 $ $ 24 $ bit depth "Cap" image

Download: Full-size image PowerPoint slide

Figure 5. The Kodak images $ 06, 09, 15 $ of $ 384 \times 256 $ pixels used in the experimentation. The second line is the enlarged images

Download: Full-size image PowerPoint slide

Figure 6. Decompression results for the Kodak images $ \rm 06, 09, $ and $ \rm 15 $. The even rows are the enlarged images

Download: Full-size image PowerPoint slide

Figure 7. Decompression results for Kodak image $ \rm 15 $ using different methods

Download: Full-size image PowerPoint slide

Figure 8. Plots of energy and PSNR values for decompressing the Kodak image $ \rm 15 $

Download: Full-size image PowerPoint slide

Figure 9. $ \rm (a) $ Original false-color image for the Los Angeles multispectral image set, which is generated by assigning the spectral band $ \rm B4 $ to red, $ \rm B3 $ to green, and $ \rm B2 $ to blue. $ \rm (b)–(f) $ Original images for the spectral bands $ \rm B1-B5 $. The size of these images is $ 512\times512 $ $ 8 $ bit depth. $ \rm (g)–(h) $ Original images for the thermal bands $ \rm B61 $ and $ \rm B62 $. The size of these two images is $ 512\times512 $ $ 8 $ bit depth

Download: Full-size image PowerPoint slide

Figure 10. $ \rm (a) $ Structure image for the Los Angeles image thermal bands (B61 and $ \rm B62) $. $ \rm (b) $ Structure image for the Los Angeles image spectral bands $ \rm B1-B5 $ with $ \alpha $ = (0.2040, 0.2028, 0.2012, 0.2097, 0.1823). $ \rm (c) $ Structure image for the Los Angeles image spectral bands $ \rm B1-B5 $ with $ \alpha = (0.1912, 0.3681, 0.1305, 0.2244, 0.0858) $

Download: Full-size image PowerPoint slide

Figure 11. Decompression results for the Los Angeles images. {(a), (c) and (e)}: Decompressed thermal band $ \rm B62 $ using the JPEG standard, the JPEG2000 standard and the proposed method, respectively. {(b), (d) and (f)}: Decompressed spectral band $ \rm B3 $ using the JPEG standard, the JPEG2000 standard and the proposed method, respectively

Download: Full-size image PowerPoint slide

Table 1. The computational time (sec) of the proposed "one-iteration method" is compared with the local-optimal strategy in [43] for extracting representative pixels

Image	Size	Number of representative pixels	Time of One-iteration method (sec)	Time of Local-optimal strategy (sec)
05	$ 384 \times 256 $	546	6.8	18.3
23	$ 768 \times 512 $	1404	15.9	62.5

| Show Table

DownLoad: CSV

Table 2. The computational time (sec) taken by different methods

Number of representative pixels	Time of reaction-diffusion equations (21) (sec)	Time of Algorithm 2 (sec))
184	15.7	3.3
394	21.6	3.6

| Show Table

DownLoad: CSV

Table 3. Decompression results of different methods. The PSNR values are shown in Table 3(a). Table 3(b) gives the SSIM values of the decompressed images. For the image to be tested, we first calculate the index value of each channel and then average the values of all channels to obtain the PSNR or SSIM values of the decompressed image. Here the notation $ \rm CR $ denotes the compression ratio

(a) PSNR
Image	CR	JPEG	JPEG2000	Shi et al.	Proposed
06	20:1	29.5755	31.5652	30.5245	30.6496
09	41:1	28.0773	30.3243	29.2732	29.5357
15	61:1	25.7227	28.3352	26.8622	27.4383
(b) SSIM
Image	CR	JPEG	JPEG2000	Shi et al.	Proposed
06	20:1	0.8387	0.8702	0.8556	0.8566
09	41:1	0.8173	0.8816	0.8704	0.8751
15	61:1	0.7247	0.8086	0.7985	0.8112

| Show Table

DownLoad: CSV

Table 4. PSNR values of the two thermal bands (B61 and B62) using different decompression methods. Here the notation $ \rm CR $ denotes the compression ratio

Thermal bands	CR	JPEG	JPEG2000	Proposed
B61	10:1	44.5696	44.7990	47.0500
B62	10:1	40.8826	41.3790	42.9845
Avg.		42.7261	43.0890	45.0172

| Show Table

DownLoad: CSV

Table 5. PSNR values of the spectral bands $ \rm B1-B5 $ using different decompression methods. Here the notation $ \rm CR $ denotes the compression ratio

Spectral bands	CR	JPEG	JPEG2000	Proposed ($\alpha$ _rand)	Proposed
B1	27:1	32.5552	32.8792	33.5227	33.4983
B2	27:1	31.6115	31.8138	34.4564	33.8015
B3	27:1	28.9824	29.4833	31.1467	31.4802
B4	27:1	30.5670	31.1113	28.8501	29.5729
B5	27:1	27.7357	28.1368	27.6438	28.6967
Avg.		30.2903	30.6848	31.1239	31.4103

| Show Table

DownLoad: CSV

Related Papers

Cited by

References

[1]	N. Amrani, J. Serra-Sagristà, P. Peter and J. Weickert, Diffusion-based inpainting for coding remote-sensing data, IEEE Geosci. Remote Sens. Lett., 14 (2017), 1203-1207. doi: 10.1109/LGRS.2017.2702106.
[2]	G. Aubert and P. Kornprobst, Mathematical Problems in Image Processing: Partial Differential Equations and The Calculus of Variations, 2$^{nd}$ edition, Springer-Verlag, New York, 2006.
[3]	M. Bertalmio, Strong-continuation, contrast-invariant inpainting with a third-order optimal PDE, IEEE Trans. Image Process., 15 (2006), 1934-1938. doi: 10.1109/TIP.2006.877067.
[4]	S. Boyd, N. Parikh, E. Chu, B. Peleato and J. Eckstein, Distributed optimization and statistical learning via the alternating direction method of multipliers, Found. Trends Mach. Learn., 3 (2011), 1-122. doi: 10.1561/2200000016.
[5]	V. Caselles, B. Coll and J.-M. Morel, Geometry and color in natural images, J. Math. Imaging Vision, 16 (2002), 89-105. doi: 10.1023/A:1013943314097.
[6]	A. Chambolle and T. Pock, A first-order primal-dual algorithm for convex problems with applications to imaging, J. Math. Imaging Vision, 40 (2011), 120-145. doi: 10.1007/s10851-010-0251-1.
[7]	T. F. Chan and J. Shen, Variational image inpainting, Commun. Pure Appl. Math., 58 (2005), 579-619. doi: 10.1002/cpa.20075.
[8]	G. Chen, Image segmentation algorithm combined with regularized PM de-noising model and improved watershed algorithm, J. Med. Imaging Health Inf., 10 (2020), 515-521. doi: 10.1166/jmihi.2020.2899.
[9]	D. Cremers, T. Kohlberger and C. Schnörr, Shape statistics in kernel space for variational image segmentation, Pattern Recognit., 36 (2003), 1929-1943. doi: 10.1016/S0031-3203(03)00056-6.
[10]	B. Dacorogna, Direct Methods in The Calculus of Variations, Springer-Verlag, New York, 2008.
[11]	A. Danielyan, V. Katkovnik and K. Egiazarian, BM3D frames and variational image deblurring, IEEE Trans. Image Process., 21 (2011), 1715-1728. doi: 10.1109/TIP.2011.2176954.
[12]	L.-J. Deng, R. Glowinski and X.-C. Tai, A new operator splitting method for the {E}uler elastica model for image smoothing, SIAM J. Imag. Sci., 12 (2019), 1190-1230. doi: 10.1137/18M1226361.
[13]	S. Di Zenzo, A note on the gradient of a multi-image, Computer Vision, Graphics, and Image Processing, 33 (1986), 116-125. doi: 10.1016/0734-189X(86)90223-9.
[14]	O. Drblíková and K. Mikula, Convergence analysis of finite volume scheme for nonlinear tensor anisotropic diffusion in image processing, SIAM J. Numer. Anal., 46 (2008), 37-60. doi: 10.1137/070685038.
[15]	J. Duran, A. Buades, B. Coll and C. Sbert, A nonlocal variational model for pansharpening image fusion, SIAM J. Imag. Sci., 7 (2014), 761-796. doi: 10.1137/130928625.
[16]	J. Eckstein and D. P. Bertsekas, On the Douglas-rachford splitting method and the proximal point algorithm for maximal monotone operators, Math. Program., 55 (1992), 293-318. doi: 10.1007/BF01581204.
[17]	J. Eckstein and M. C. Ferris, Operator-splitting methods for monotone affine variational inequalities, with a parallel application to optimal control, Informs J. Comput., 10 (1998), 218-235. doi: 10.1287/ijoc.10.2.218.
[18]	M. J. Ehrhardt and S. R. Arridge, Vector-valued image processing by parallel level sets, IEEE Trans. Image Process., 23 (2014), 9-18. doi: 10.1109/TIP.2013.2277775.
[19]	R. Franzen, Kodak lossless true color image suite, 1999. Available from: http://r0k.us/graphics/kodak/.
[20]	I. Galić, J. Weickert, M. Welk, A. Bruhn, A. Belyaev and H.-P. Seidel, Towards PDE-based image compression, International Workshop on Variational, Geometric, and Level Set Methods in Computer Vision, Springer, Berlin, Heidelberg, 2005, 37–48. doi: 10.1007/11567646_4.
[21]	I. Galić, J. Weickert, M. Welk, A. Bruhn, A. Belyaev and H.-P. Seidel, Image compression with anisotropic diffusion, J. Math. Imaging Vision, 31 (2008), 255-269. doi: 10.1007/s10851-008-0087-0.
[22]	T. Goldstein and S. Osher, The split bregman method for $L_{1}$-regularized problems, SIAM J. Imag. Sci., 2 (2009), 323-343. doi: 10.1137/080725891.
[23]	H. Grossauer, A combined PDE and texture synthesis approach to inpainting, European Conference on Computer Vision, Springer, Berlin, Heidelberg, 2004,214–224. doi: 10.1007/978-3-540-24671-8_17.
[24]	R. M. Haralick, K. Shanmugam and I. H. Dinstein, Textural features for image classification, IEEE Trans. Syst. Man Cybern., 6 (1973), 610-621. doi: 10.1109/TSMC.1973.4309314.
[25]	M. Kreuter, Sobolev Spaces of Vector-Valued Functions, Master thesis, Ulm University, 2015.
[26]	E. Kreyszig, Introduction to Differential Geometry and Riemannian Geometry, University of Toronto Press, Toronto, 1968.
[27]	S. Lee, S.-W. Park, P. Oh and M. G. Kang, Colorization-based compression using optimization, IEEE Trans. Image Process., 22 (2013), 2627-2636. doi: 10.1109/TIP.2013.2253486.
[28]	A. S. Lewis and G. Knowles, Image compression using the 2-D wavelet transform, IEEE Trans. Image Process., 1 (1992), 244-250. doi: 10.1109/83.136601.
[29]	J.-M. Morel and S. Solimini, Variational Methods in Image Segmentation: With Seven Image Processing Experiments, Birkhäuser, Boston, 1995. doi: 10.1007/978-1-4684-0567-5.
[30]	K. T. Mullen, The contrast sensitivity of human colour vision to red-green and blue-yellow chromatic gratings, J. Physiol., 359 (1985), 381-400. doi: 10.1113/jphysiol.1985.sp015591.
[31]	H.-H. Nagel, Constraints for the estimation of displacement vector fields from image sequences, IJCAI, 1983,945–951.
[32]	S. Ono, T. Miyata and Y. Sakai, Colorization-based coding by focusing on characteristics of colorization bases, 28th Picture Coding Symposium, Nagoya, Japan, 2010,230–233. doi: 10.1109/PCS.2010.5702473.
[33]	W. B. Pennebaker and J. L. Mitchell, JPEG: Still Image Data Compression Standard, Springer Science & Business Media, 1992.
[34]	P. Perona and J. Malik, Scale-space and edge detection using anisotropic diffusion, IEEE Trans. Pattern Anal. Mach. Intell., 12 (1990), 629-639. doi: 10.1109/34.56205.
[35]	P. Peter, L. Kaufhold and J. Weickert, Turning diffusion-based image colorization into efficient color compression, IEEE Trans. Image Process., 26 (2017), 860-869. doi: 10.1109/TIP.2016.2627800.
[36]	P. Petersen, Riemannian Geometry, 2$^{nd}$ edition, Springer-Verlag, New York 2006.
[37]	M. Rabbani, JPEG2000: Image compression fundamentals, standards and practice, J. Electron. Imaging, 11 (2002), 286. doi: 10.1117/1.1469618.
[38]	M. Renardy and R. C. Rogers, An Introduction to Partial Differential Equations, 2$^{nd}$ edition, Springer-Verlag, New York, 2004.
[39]	L. I. Rudin, S. Osher and E. Fatemi, Nonlinear total variation based noise removal algorithms, Physica D: Nonlinear Phenomena, 60 (1992), 259-268. doi: 10.1016/0167-2789(92)90242-F.
[40]	G. Sapiro and D. L. Ringach, Anisotropic diffusion of multivalued images with applications to color filtering, IEEE Trans. Image Process., 5 (1996), 1582-1586. doi: 10.1109/83.541429.
[41]	C. Schmaltz, P. Peter, M. Mainberger, F. Ebel, J. Weickert and A. Bruhn, Understanding, optimising, and extending data compression with anisotropic diffusion, Int. J. Comput. Vision, 108 (2014), 222-240. doi: 10.1007/s11263-014-0702-z.
[42]	K. Shi and Z. Guo, On the existence of weak solutions for a curvature driven elliptic system applied to image inpainting, Appl. Math. Lett., 99 (2020), 106003. doi: 10.1016/j.aml.2019.106003.
[43]	K. Shi, D. Zhang, Z. Guo and B. Wu, A linear reaction-diffusion system with interior degeneration for color image compression, SIAM J. Imag. Sci., 11 (2018), 442-472. doi: 10.1137/17M1137991.
[44]	S. S. Swami and A. O. Mulani, An efficient FPGA implementation of discrete wavelet transform for image compression, 2017 International Conference on Energy, Communication, Data Analytics and Soft Computing (ICECDS), Chennai, India, 2017, 3385–3389. doi: 10.1109/ICECDS.2017.8390088.
[45]	X.-C. Tai, J. Hahn and G. J. Chung, A fast algorithm for Euler's elastica model using augmented Lagrangian method, SIAM J. Imag. Sci., 4 (2011), 313-344. doi: 10.1137/100803730.
[46]	C. Tian and Y. Chen, Image segmentation and denoising algorithm based on partial differential equations, IEEE Sens. J., 20 (2020), 11935-11942. doi: 10.1109/JSEN.2019.2959704.
[47]	A. N. Tikhonov and V. Y. Arsenin, Solutions of Ill-Posed Problems, Winston, New York, 1997.
[48]	K. Uruma, K. Konishi, T. Takahashi and T. Furukawa, Fast colorization based image coding algorithm using multiple resolution images, EURASIP J. Image Video Process, 1 (2016), 1-15. doi: 10.1186/s13640-016-0107-8.
[49]	Y. Wang, W. Yin and J. Zeng, Global convergence of ADMM in nonconvex nonsmooth optimization, J. Sci. Comput., 78 (2019), 29-63. doi: 10.1007/s10915-018-0757-z.
[50]	Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, Image quality assessment: From error visibility to structural similarity, IEEE Trans. Image Process., 13 (2004), 600-612. doi: 10.1109/TIP.2003.819861.
[51]	J. Weickert, Theoretical foundations of anisotropic diffusion in image processing, in Theoretical Foundations of Computer Vision, Springer, Vienna, 1996,221–236. doi: 10.1007/978-3-7091-6586-7_13.
[52]	J. Weickert, Anisotropic Diffusion in Image Processing, Teubner Stuttgart, 1998.
[53]	J. Weickert and H. Scharr, A scheme for coherence-enhancing diffusion filtering with optimized rotation invariance, J. Visual Commun. Image Represent., 13 (2002), 103-118. doi: 10.1006/jvci.2001.0495.
[54]	J. Weickert, M. Welk and M. Wickert, $L ^{2}$-stable nonstandard finite differences for anisotropic diffusion, International Conference on Scale Space and Variational Methods in Computer Vision, Springer, Berlin, Heidelberg, 2013,380–391. doi: 10.1007/978-3-642-38267-3_32.