ZS [52] | SHP [39] | Proposed | |
Figure 4(b) | 1.3799 | 0.6495 | 0.5308 |
Figure 5(e) | 0.4043 | 0.3753 | 0.3751 |
Figure 5(f) | 0.3959 | 0.3950 | 0.3684 |
Figure 5(g) | 0.4811 | 0.4455 | 0.3064 |
Figure 5(h) | 0.4823 | 0.4756 | 0.3418 |
In this paper, we focus on the challenging problem of removing the spatially varying out-of-focus blur from a single natural image. We first propose an effective method to estimate the blur map by the total variation refinement on Hölder coefficient, then discuss the properties of the corresponding kernel matrix. A tight-frame based energy functional, whose minimizer is related to the optimal defocus result, is thus built. For tackling functional more efficiently, we describe the numerical procedure based on an accelerated primal-dual scheme. To verify the effectiveness of our method, we compare it with some state-of-the-art schemes using both synthesized and natural images. Experimental results demonstrate that the proposed method performs better than the compared methods.
Citation: |
Figure 3. The blur maps of Figure 1(a) ($800\times 600$ pixels): (a) the rough blur map $\boldsymbol{\tilde{\sigma}}$; (b) the refined blur map $\boldsymbol{{\sigma}}$.
Figure 4. (a) a sharp image (size:100$\times$ 400); (b) a blur version of (a) (the blur levels are 1 and 4 for 171th-230th and 271th-330th columns, respectively); (c)-(e) the blur maps genrated by ZS [52], SHP [39] and proposed methods; (f) the ground truth of blur map; (g) corresponding profiles of 1-st line for (c)-(f).
Figure 5. (a) and (b) are original images; (c) and (d) are two blur maps (range: 1-5); (e): the blurred version of (a) using (c) as blur map; (f): the blurred version of (a) using (d) as blur map; (g): the blurred version of (b) using (c) as blur map; (h): the blurred version of (b) using (d) as blur map.
Figure 6. Synthesised experiments: (A1-A5)/(B1-B5) are the deblurred results (with zoomed regions) of Figure 5(e)/(f) by Matlab, XJ [49], CJLS [8], SHP [39], and proposed methods, respectively.
Figure 7. Synthesised experiments: (A1-A5)/(B1-B5) are the deblurred results (with zoomed regions) of Figure 5(g)/(h) by Matlab, XJ [49], CJLS [8], SHP [39], and proposed methods, respectively.
ZS [52] | SHP [39] | Proposed | |
Figure 4(b) | 1.3799 | 0.6495 | 0.5308 |
Figure 5(e) | 0.4043 | 0.3753 | 0.3751 |
Figure 5(f) | 0.3959 | 0.3950 | 0.3684 |
Figure 5(g) | 0.4811 | 0.4455 | 0.3064 |
Figure 5(h) | 0.4823 | 0.4756 | 0.3418 |
Table 2. Time comparison with other methods (second).
Figure 5(e) | Figure 5(f) | Figure 5(g) | Figure 5(h) | Figure 9(a) | Figure 10(a) | Figure 11(a) | |
image size | 300 × 286 | 300 × 286 | 265 × 300 | 265 × 300 | 200 × 300 | 800 × 600 | 534 × 800 |
Matlab | 1.18 | 1.22 | 1.00 | 1.17 | 2.03 | 6.70 | 6.00 |
XJ [49](C) | 6.45 | 6.55 | 6.36 | 6.35 | 11.43 | 18.01 | 16.95 |
CJLS [8] | 86.59 | 83.74 | 88.27 | 89.84 | 194.95 | 780.30 | 693.77 |
SHP [39] | 70.21 | 69.40 | 72.26 | 71.55 | 126.37 | 187.01 | 168.68 |
Proposed | 38.33 | 37.60 | 34.98 | 36.46 | 93.45 | 137.93 | 141.32 |
S. Bae
and F. Durand
, Defocus magnification, in Computer Graphics Forum, 26 (2007)
, 571-579.
doi: 10.1111/j.1467-8659.2007.01080.x.![]() ![]() |
|
L. Bar
, N. Sochen
and N. Kiryati
, Restoration of images with piecewise space-variant blur, in In Proceedings of the First International Conference on Scale Space Methods and Variational Methods in Computer Vision, 4485 (2007)
, 533-544.
doi: 10.1007/978-3-540-72823-8_46.![]() ![]() |
|
G. Blanchet
and L. Moisan
, An explicit sharpness index related to global phase coherence, in 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), (2012)
, 1065-1068.
doi: 10.1109/ICASSP.2012.6288070.![]() ![]() |
|
C. Boor,
A Practical Guide to Splines vol. 27, 1st edition, New York: Springer, 1987.
![]() |
|
S. Boyd
, N. Parikh
, E. Chu
, B. Peleato
and J. Eckstein
, Distributed optimization and statistical learning via the alternating direction method of multipliers, Foundations and Trends in Machine Learning, 3 (2011)
, 1-122.
doi: 10.1561/2200000016.![]() ![]() |
|
C. Byrne
, Bounds on the largest singular value of a matrix and the convergence of simultaneous and block-iterative algorithms for sparse linear systems, International Transactions in Operational Research, 16 (2009)
, 465-479.
doi: 10.1111/j.1475-3995.2009.00692.x.![]() ![]() ![]() |
|
J. Cai
, R. Chan
and Z. Shen
, A framelet-based image inpainting algorithm, Applied Computational Harmonic Analysis, 24 (2008)
, 131-149.
doi: 10.1016/j.acha.2007.10.002.![]() ![]() ![]() |
|
J. Cai
, H. Ji
, C. Liu
and Z. Shen
, Framelet-based blind motion deblurring from a single image, IEEE Transactions on Image Processing, 21 (2012)
, 562-572.
doi: 10.1109/TIP.2011.2164413.![]() ![]() ![]() |
|
J. Cai
, S. Osher
and Z. Shen
, Linearized bregman iterations for frame-based image deblurring, SIAM Journal on Imaging Sciences, 2 (2009)
, 226-252.
doi: 10.1137/080733371.![]() ![]() ![]() |
|
J. Cai
, S. Osher
and Z. Shen
, Split bregman method and frame based image restoration, Notices of the American Mathematical Society, 8 (2009)
, 337-369.
![]() ![]() |
|
A. Chai
and Z. Shen
, Deconvolution: A wavelet frame approach, Numerische Mathematik, 106 (2007)
, 529-587.
doi: 10.1007/s00211-007-0075-0.![]() ![]() ![]() |
|
A. Chakrabarti
, T. Zickler
and W. Freeman
, Analyzing spatially-varying blur, in Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, (2010)
, 2512-2519.
doi: 10.1109/CVPR.2010.5539954.![]() ![]() |
|
A. Chambolle
and A. Pock
, A first-order primal-dual algorithm for convex problems with applications to imaging, Journal of Mathematical Imaging and Vision, 40 (2011)
, 120-145.
doi: 10.1007/s10851-010-0251-1.![]() ![]() ![]() |
|
R. Chan
, S. Riemenschneider
, L. Shen
and Z. Shen
, Tight frame: an efficient way for high-resolution image reconstruction, Applied Computational Harmonic Analysis, 17 (2004)
, 91-115.
doi: 10.1016/j.acha.2004.02.003.![]() ![]() ![]() |
|
S. Chan
and T. Nguyen
, Single image spatially variant out-of-focus blur removal, in Image Processing (ICIP), 2011 18th IEEE International Conference on, (2011)
, 677-680.
![]() |
|
T. Chan,
Theory and Computation of Variational Image Deblurring Lecture notes series, WSPC, 2005.
![]() |
|
Y. Chen
, G. Lan
and Y. Ouyang
, Optimal primal-dual methods for a class of saddle point problems, SIAM Journal on Optimization, 24 (2014)
, 1779-1814.
doi: 10.1137/130919362.![]() ![]() ![]() |
|
I. Daubechies
, B. Han
, A. Ron
and Z. Shen
, Framelets: Mra-based constructions of wavelet frames, Applied and Computational Harmonic Analysis, 14 (2003)
, 1-46.
doi: 10.1016/S1063-5203(02)00511-0.![]() ![]() ![]() |
|
I. Ekeland and R. Témam,
Convex Analysis and Variational Problems Classics in Applied Mathematics (Book 28), Society for Industrial and Applied Mathematics, Philadelphia, 1999.
![]() ![]() |
|
L. Evans, Partial Differential Equations, vol. 19, chapter Holder-spaces, 456–461, American
Mathematical Society, 1998.
![]() |
|
F. Fang
, G. Zhang
, F. Li
and C. Shen
, Framelet based pan-sharpening via a variational method, Neurocomputing, 129 (2014)
, 362-377.
doi: 10.1016/j.neucom.2013.09.022.![]() ![]() |
|
E. Faramarzi
, D. Rajan
and M. Christensen
, Unified blind method for multi-image super-resolution and single/multi-image blur deconvolution, IEEE Transactions on Image Processing, 22 (2013)
, 2101-2114.
doi: 10.1109/TIP.2013.2237915.![]() ![]() ![]() |
|
T. Goldstein
and S. Osher
, The split bregman method for L1-regularized problems, SIAM Journal on Imaging Sciences, 2 (2009)
, 323-343.
doi: 10.1137/080725891.![]() ![]() ![]() |
|
R. Gonzalez and R. Woods,
Digital Image Processing 3rd edition, Prentice-Hall, Inc. , Upper Saddle River, NJ, USA, 2006.
![]() |
|
A. Gupta
, N. Joshi
, L. Zitnick
, M. Cohen
and B. Curless
, Single image deblurring using motion density functions, in ECCV '10: Proceedings of the 10th European Conference on Computer Vision, 6311 (2010)
, 171-184.
doi: 10.1007/978-3-642-15549-9_13.![]() ![]() |
|
R. Horn and C. Johnson,
Matrix Analysis Cambridge University Press, Cambridge, 1985.
![]() ![]() |
|
Y. Huang
, D. Lu
and T. Zeng
, A two-step approach for the restoration of images corrupted by multiplicative, SIAM Journal on Scientific Computing, 35 (2013)
, 2856-2873.
doi: 10.1137/120898693.![]() ![]() ![]() |
|
S. Jaffard
, Wavelet techniques in multifractal analysis, in In Proceedings of symposia in pure mathematics, 72 (2004)
, 91-152.
![]() ![]() |
|
S. Kim, I. Eom and Y. Kim,
Image Interpolation Based on Statistical Relationship Between Wavelet Subbands in Multimedia and Expo, 2007 IEEE International Conference on, 2007.
doi: 10.1109/ICME.2007.4285002.![]() ![]() |
|
S. Kindermann
, S. Osher
and P. Jones
, Deblurring and denoising of images by nonlocal functionals, SIAM Multiscale Modeling and Simulation, 4 (2005)
, 1091-1115.
doi: 10.1137/050622249.![]() ![]() ![]() |
|
P. Legrand
and J. Vehel
, Local regularity-based image denoising, in Image Processing, 2003. ICIP 2003. Proceedings. 2003 International Conference on, 3 (2003)
, 377-380.
![]() |
|
Y. Lou
, E. Esser
, H. Zhao
and J. Xin
, Partially blind deblurring of barcode from out-of-focus blur, SIAM Journal on Imaging Sciences, 7 (2014)
, 740-760.
doi: 10.1137/130931254.![]() ![]() ![]() |
|
Y. Lou
, X. Zhang
, S. Osher
and A. Bertozzi
, Image recovery via nonlocal operators, Journal of Scientific Computing, 42 (2010)
, 185-197.
doi: 10.1007/s10915-009-9320-2.![]() ![]() ![]() |
|
J. Muzy
, E. Bacry
and A. Arneodo
, The multifractal formalism revisited with wavelets, International Journal of Bifurcation and Chaos, 4 (1994)
, 245-302.
doi: 10.1142/S0218127494000204.![]() ![]() ![]() |
|
J. Nagy
and D. O'Leary
, Restoring images degraded by spatially variant blur, SIAM Journal on Scientific Computing, 19 (1998)
, 1063-1082.
doi: 10.1137/S106482759528507X.![]() ![]() ![]() |
|
J. Oliveira
, M. Figueiredo
and J. Bioucas-Dias
, Parametric blur estimation for blind restoration of natural images: Linear motion and out-of-focus, IEEE Transactions on Image Processing, 23 (2014)
, 466-477.
doi: 10.1109/TIP.2013.2286328.![]() ![]() ![]() |
|
A. Ron
and Z. Shen
, Affine system in $l_2(\mathbb{R}^d)$: The analysis of the analysis operator, Journal of Functional Analysis, 148 (1997)
, 408-447.
doi: 10.1006/jfan.1996.3079.![]() ![]() ![]() |
|
L. Rudin
, S. Osher
and E. Fatemi
, Nonlinear total variation based noise removal algorithms, Physica D, 60 (1992)
, 259-268.
doi: 10.1016/0167-2789(92)90242-F.![]() ![]() ![]() |
|
C. Shen
, W. Hwang
and S. Pei
, Spatially-varying out-of-focus image deblurring with l1-2 optimization and a guided blur map, in Acoustics, Speech and Signal Processing (ICASSP), 2012 IEEE International Conference on, (2012)
, 1069-1072.
doi: 10.1109/ICASSP.2012.6288071.![]() ![]() |
|
Y. Tai
and M. Brown
, Single image defocus map estimation using local contrast prior, in Proceedings of the 16th IEEE International Conference on Image Processing, ICIP'09, (2009)
, 1777-1780.
![]() |
|
H. Trussell
and S. Fogel
, Identification and restoration of spatially variant motion blurs in sequential images, IEEE Transactions on Image Processing, 1 (1992)
, 123-126.
doi: 10.1109/83.128039.![]() ![]() |
|
M. Unser
and T. Blu
, Mathematical properties of the {JPEG2000} wavelet filters, IEEE Transactions on Image Processing, 12 (2003)
, 1080-1090.
doi: 10.1109/TIP.2003.812329.![]() ![]() ![]() |
|
J. Véhel, Fractals in Multimedia, vol. 132 of The IMA Volumes in Mathematics and its
Application, chapter Signal Enhancement Based on Hölder Regularity Analysis, 197–209,
Springer, New York, 2002.
![]() |
|
Y. Wang, W. Yin and Y. Zhang,
A Fast Algorithm for Image Deblurring with Total Variation Regularization CAAM technical reports, 2007.
![]() |
|
Z. Wang
, A. Bovik
, H. Sheikh
and E. Simoncelli
, Image quality assessment: from error visibility to structural similarity, IEEE Transactions on Image Processing, 13 (2004)
, 600-612.
doi: 10.1109/TIP.2003.819861.![]() ![]() |
|
O. Whyte
, J. Sivic
, A. Zisserman
and J. Ponce
, Non-uniform deblurring for shaken images, Int. J. Comput. Vis., 98 (2012)
, 168-186.
doi: 10.1007/s11263-011-0502-7.![]() ![]() ![]() |
|
C. Wu
and X. Tai
, Augmented lagrangian method, dual methods and split bregman iteration for {ROF} model, in Scale Space and Variational Methods in Computer Vision, (2009)
, 502-513.
![]() |
|
J. Xu
, H. Chang
and J. Qin
, Domain decomposition method for image deblurring, Journal of Computational and Applied Mathematics, 271 (2014)
, 401-414.
doi: 10.1016/j.cam.2014.03.030.![]() ![]() ![]() |
|
L. Xu
and J. Jia
, Two-phase kernel estimation for robust motion deblurring, Computer Vision C ECCV 2010: The series Lecture Notes in Computer Science, 6311 (2010)
, 157-170.
doi: 10.1007/978-3-642-15549-9_12.![]() ![]() |
|
L. Xu and J. Jia, Two-phase kernel estimation for robust motion deblurring http://www.cse.cuhk.edu.hk/leojia/projects/robust_deblur/, 2010.
doi: 10.1007/978-3-642-15549-9_12.![]() ![]() |
|
X. Zhu
, S. Cohen
, S. Schiller
and P. Milanfar
, Estimating spatially varying defocus blur from a single image, IEEE Transactions on Image Processing, 22 (2013)
, 4879-4891.
doi: 10.1109/TIP.2013.2279316.![]() ![]() ![]() |
|
S. Zhuo
and T. Sim
, Defocus map estimation from a single image, Pattern Recognition, 44 (2011)
, 1852-1858.
doi: 10.1016/j.patcog.2011.03.009.![]() ![]() |
The spatially-varying blurring images.
The distributions of the NHC for in-focus (red) and blur images (blue).
The blur maps of Figure 1(a) (
(a) a sharp image (size:100
(a) and (b) are original images; (c) and (d) are two blur maps (range: 1-5); (e): the blurred version of (a) using (c) as blur map; (f): the blurred version of (a) using (d) as blur map; (g): the blurred version of (b) using (c) as blur map; (h): the blurred version of (b) using (d) as blur map.
Synthesised experiments: (A1-A5)/(B1-B5) are the deblurred results (with zoomed regions) of Figure 5(e)/(f) by Matlab, XJ [49], CJLS [8], SHP [39], and proposed methods, respectively.
Synthesised experiments: (A1-A5)/(B1-B5) are the deblurred results (with zoomed regions) of Figure 5(g)/(h) by Matlab, XJ [49], CJLS [8], SHP [39], and proposed methods, respectively.
PSNR, SSIM, and SI values corresponding to each figure.
Original image and its deblurred results with zoomed regions. (a) The original image (RGB,
Original image and its deblurred results with zoomed regions. (a) The original image (RGB,
Original image and its deblurred results with zoomed regions. (a) The original image (RGB,