Advanced Search
Article Contents
Article Contents

Reducing spatially varying out-of-focus blur from natural image

  • * Corresponding author: Tieyong Zeng

    * Corresponding author: Tieyong Zeng
Abstract Full Text(HTML) Figure(11) / Table(2) Related Papers Cited by
  • In this paper, we focus on the challenging problem of removing the spatially varying out-of-focus blur from a single natural image. We first propose an effective method to estimate the blur map by the total variation refinement on Hölder coefficient, then discuss the properties of the corresponding kernel matrix. A tight-frame based energy functional, whose minimizer is related to the optimal defocus result, is thus built. For tackling functional more efficiently, we describe the numerical procedure based on an accelerated primal-dual scheme. To verify the effectiveness of our method, we compare it with some state-of-the-art schemes using both synthesized and natural images. Experimental results demonstrate that the proposed method performs better than the compared methods.

    Mathematics Subject Classification: Primary: 68U10, 65K10, 35A15, 91-08; Secondary: 62H35.


    \begin{equation} \\ \end{equation}
  • 加载中
  • Figure 1.  The spatially-varying blurring images.

    Figure 2.  The distributions of the NHC for in-focus (red) and blur images (blue).

    Figure 3.  The blur maps of Figure 1(a) ($800\times 600$ pixels): (a) the rough blur map $\boldsymbol{\tilde{\sigma}}$; (b) the refined blur map $\boldsymbol{{\sigma}}$.

    Figure 4.  (a) a sharp image (size:100$\times$ 400); (b) a blur version of (a) (the blur levels are 1 and 4 for 171th-230th and 271th-330th columns, respectively); (c)-(e) the blur maps genrated by ZS [52], SHP [39] and proposed methods; (f) the ground truth of blur map; (g) corresponding profiles of 1-st line for (c)-(f).

    Figure 5.  (a) and (b) are original images; (c) and (d) are two blur maps (range: 1-5); (e): the blurred version of (a) using (c) as blur map; (f): the blurred version of (a) using (d) as blur map; (g): the blurred version of (b) using (c) as blur map; (h): the blurred version of (b) using (d) as blur map.

    Figure 6.  Synthesised experiments: (A1-A5)/(B1-B5) are the deblurred results (with zoomed regions) of Figure 5(e)/(f) by Matlab, XJ [49], CJLS [8], SHP [39], and proposed methods, respectively.

    Figure 7.  Synthesised experiments: (A1-A5)/(B1-B5) are the deblurred results (with zoomed regions) of Figure 5(g)/(h) by Matlab, XJ [49], CJLS [8], SHP [39], and proposed methods, respectively.

    Figure 8.  PSNR, SSIM, and SI values corresponding to each figure.

    Figure 9.  Original image and its deblurred results with zoomed regions. (a) The original image (RGB, $296\times 877$ pixels); (b)-(f) the results of Matlab, XJ [49], CJLS [8], SHP [39], and the proposed methods.

    Figure 10.  Original image and its deblurred results with zoomed regions. (a) The original image (RGB, $800\times 600$ pixels); (b)-(f) the results of Matlab, XJ [49], CJLS [8], SHP [39], and the proposed methods.

    Figure 11.  Original image and its deblurred results with zoomed regions. (a) The original image (RGB, $550\times 760$ pixels); (b)-(f) the results of Matlab, XJ [49], CJLS [8], SHP [39], and the proposed methods.

    Table 1.  The MAE value of each blur map generated by ZS [52], SHP [39] and Proposed methods.

    ZS [52]SHP [39]Proposed
    Figure 4(b)1.37990.64950.5308
    Figure 5(e)0.40430.37530.3751
    Figure 5(f)0.39590.39500.3684
    Figure 5(g)0.48110.44550.3064
    Figure 5(h)0.48230.47560.3418
     | Show Table
    DownLoad: CSV

    Table 2.  Time comparison with other methods (second).

    Figure 5(e)Figure 5(f)Figure 5(g)Figure 5(h)Figure 9(a)Figure 10(a)Figure 11(a)
    image size300 × 286300 × 286265 × 300265 × 300200 × 300800 × 600534 × 800
    XJ [49](C)6.456.556.366.3511.4318.0116.95
    CJLS [8]86.5983.7488.2789.84194.95780.30693.77
    SHP [39]70.2169.4072.2671.55126.37187.01168.68
     | Show Table
    DownLoad: CSV
  •   S. Bae  and  F. Durand , Defocus magnification, in Computer Graphics Forum, 26 (2007) , 571-579.  doi: 10.1111/j.1467-8659.2007.01080.x.
      L. Bar , N. Sochen  and  N. Kiryati , Restoration of images with piecewise space-variant blur, in In Proceedings of the First International Conference on Scale Space Methods and Variational Methods in Computer Vision, 4485 (2007) , 533-544.  doi: 10.1007/978-3-540-72823-8_46.
      G. Blanchet  and  L. Moisan , An explicit sharpness index related to global phase coherence, in 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), (2012) , 1065-1068.  doi: 10.1109/ICASSP.2012.6288070.
      C. Boor, A Practical Guide to Splines vol. 27, 1st edition, New York: Springer, 1987.
      S. Boyd , N. Parikh , E. Chu , B. Peleato  and  J. Eckstein , Distributed optimization and statistical learning via the alternating direction method of multipliers, Foundations and Trends in Machine Learning, 3 (2011) , 1-122.  doi: 10.1561/2200000016.
      C. Byrne , Bounds on the largest singular value of a matrix and the convergence of simultaneous and block-iterative algorithms for sparse linear systems, International Transactions in Operational Research, 16 (2009) , 465-479.  doi: 10.1111/j.1475-3995.2009.00692.x.
      J. Cai , R. Chan  and  Z. Shen , A framelet-based image inpainting algorithm, Applied Computational Harmonic Analysis, 24 (2008) , 131-149.  doi: 10.1016/j.acha.2007.10.002.
      J. Cai , H. Ji , C. Liu  and  Z. Shen , Framelet-based blind motion deblurring from a single image, IEEE Transactions on Image Processing, 21 (2012) , 562-572.  doi: 10.1109/TIP.2011.2164413.
      J. Cai , S. Osher  and  Z. Shen , Linearized bregman iterations for frame-based image deblurring, SIAM Journal on Imaging Sciences, 2 (2009) , 226-252.  doi: 10.1137/080733371.
      J. Cai , S. Osher  and  Z. Shen , Split bregman method and frame based image restoration, Notices of the American Mathematical Society, 8 (2009) , 337-369. 
      A. Chai  and  Z. Shen , Deconvolution: A wavelet frame approach, Numerische Mathematik, 106 (2007) , 529-587.  doi: 10.1007/s00211-007-0075-0.
      A. Chakrabarti , T. Zickler  and  W. Freeman , Analyzing spatially-varying blur, in Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, (2010) , 2512-2519.  doi: 10.1109/CVPR.2010.5539954.
      A. Chambolle  and  A. Pock , A first-order primal-dual algorithm for convex problems with applications to imaging, Journal of Mathematical Imaging and Vision, 40 (2011) , 120-145.  doi: 10.1007/s10851-010-0251-1.
      R. Chan , S. Riemenschneider , L. Shen  and  Z. Shen , Tight frame: an efficient way for high-resolution image reconstruction, Applied Computational Harmonic Analysis, 17 (2004) , 91-115.  doi: 10.1016/j.acha.2004.02.003.
      S. Chan  and  T. Nguyen , Single image spatially variant out-of-focus blur removal, in Image Processing (ICIP), 2011 18th IEEE International Conference on, (2011) , 677-680. 
      T. Chan, Theory and Computation of Variational Image Deblurring Lecture notes series, WSPC, 2005.
      Y. Chen , G. Lan  and  Y. Ouyang , Optimal primal-dual methods for a class of saddle point problems, SIAM Journal on Optimization, 24 (2014) , 1779-1814.  doi: 10.1137/130919362.
      I. Daubechies , B. Han , A. Ron  and  Z. Shen , Framelets: Mra-based constructions of wavelet frames, Applied and Computational Harmonic Analysis, 14 (2003) , 1-46.  doi: 10.1016/S1063-5203(02)00511-0.
      I. Ekeland and R. Témam, Convex Analysis and Variational Problems Classics in Applied Mathematics (Book 28), Society for Industrial and Applied Mathematics, Philadelphia, 1999.
      L. Evans, Partial Differential Equations, vol. 19, chapter Holder-spaces, 456–461, American Mathematical Society, 1998.
      F. Fang , G. Zhang , F. Li  and  C. Shen , Framelet based pan-sharpening via a variational method, Neurocomputing, 129 (2014) , 362-377.  doi: 10.1016/j.neucom.2013.09.022.
      E. Faramarzi , D. Rajan  and  M. Christensen , Unified blind method for multi-image super-resolution and single/multi-image blur deconvolution, IEEE Transactions on Image Processing, 22 (2013) , 2101-2114.  doi: 10.1109/TIP.2013.2237915.
      T. Goldstein  and  S. Osher , The split bregman method for L1-regularized problems, SIAM Journal on Imaging Sciences, 2 (2009) , 323-343.  doi: 10.1137/080725891.
      R. Gonzalez and R. Woods, Digital Image Processing 3rd edition, Prentice-Hall, Inc. , Upper Saddle River, NJ, USA, 2006.
      A. Gupta , N. Joshi , L. Zitnick , M. Cohen  and  B. Curless , Single image deblurring using motion density functions, in ECCV '10: Proceedings of the 10th European Conference on Computer Vision, 6311 (2010) , 171-184.  doi: 10.1007/978-3-642-15549-9_13.
      R. Horn and C. Johnson, Matrix Analysis Cambridge University Press, Cambridge, 1985.
      Y. Huang , D. Lu  and  T. Zeng , A two-step approach for the restoration of images corrupted by multiplicative, SIAM Journal on Scientific Computing, 35 (2013) , 2856-2873.  doi: 10.1137/120898693.
      S. Jaffard , Wavelet techniques in multifractal analysis, in In Proceedings of symposia in pure mathematics, 72 (2004) , 91-152. 
      S. Kim, I. Eom and Y. Kim, Image Interpolation Based on Statistical Relationship Between Wavelet Subbands in Multimedia and Expo, 2007 IEEE International Conference on, 2007. doi: 10.1109/ICME.2007.4285002.
      S. Kindermann , S. Osher  and  P. Jones , Deblurring and denoising of images by nonlocal functionals, SIAM Multiscale Modeling and Simulation, 4 (2005) , 1091-1115.  doi: 10.1137/050622249.
      P. Legrand  and  J. Vehel , Local regularity-based image denoising, in Image Processing, 2003. ICIP 2003. Proceedings. 2003 International Conference on, 3 (2003) , 377-380. 
      Y. Lou , E. Esser , H. Zhao  and  J. Xin , Partially blind deblurring of barcode from out-of-focus blur, SIAM Journal on Imaging Sciences, 7 (2014) , 740-760.  doi: 10.1137/130931254.
      Y. Lou , X. Zhang , S. Osher  and  A. Bertozzi , Image recovery via nonlocal operators, Journal of Scientific Computing, 42 (2010) , 185-197.  doi: 10.1007/s10915-009-9320-2.
      J. Muzy , E. Bacry  and  A. Arneodo , The multifractal formalism revisited with wavelets, International Journal of Bifurcation and Chaos, 4 (1994) , 245-302.  doi: 10.1142/S0218127494000204.
      J. Nagy  and  D. O'Leary , Restoring images degraded by spatially variant blur, SIAM Journal on Scientific Computing, 19 (1998) , 1063-1082.  doi: 10.1137/S106482759528507X.
      J. Oliveira , M. Figueiredo  and  J. Bioucas-Dias , Parametric blur estimation for blind restoration of natural images: Linear motion and out-of-focus, IEEE Transactions on Image Processing, 23 (2014) , 466-477.  doi: 10.1109/TIP.2013.2286328.
      A. Ron  and  Z. Shen , Affine system in $l_2(\mathbb{R}^d)$: The analysis of the analysis operator, Journal of Functional Analysis, 148 (1997) , 408-447.  doi: 10.1006/jfan.1996.3079.
      L. Rudin , S. Osher  and  E. Fatemi , Nonlinear total variation based noise removal algorithms, Physica D, 60 (1992) , 259-268.  doi: 10.1016/0167-2789(92)90242-F.
      C. Shen , W. Hwang  and  S. Pei , Spatially-varying out-of-focus image deblurring with l1-2 optimization and a guided blur map, in Acoustics, Speech and Signal Processing (ICASSP), 2012 IEEE International Conference on, (2012) , 1069-1072.  doi: 10.1109/ICASSP.2012.6288071.
      Y. Tai  and  M. Brown , Single image defocus map estimation using local contrast prior, in Proceedings of the 16th IEEE International Conference on Image Processing, ICIP'09, (2009) , 1777-1780. 
      H. Trussell  and  S. Fogel , Identification and restoration of spatially variant motion blurs in sequential images, IEEE Transactions on Image Processing, 1 (1992) , 123-126.  doi: 10.1109/83.128039.
      M. Unser  and  T. Blu , Mathematical properties of the {JPEG2000} wavelet filters, IEEE Transactions on Image Processing, 12 (2003) , 1080-1090.  doi: 10.1109/TIP.2003.812329.
      J. Véhel, Fractals in Multimedia, vol. 132 of The IMA Volumes in Mathematics and its Application, chapter Signal Enhancement Based on Hölder Regularity Analysis, 197–209, Springer, New York, 2002.
      Y. Wang, W. Yin and Y. Zhang, A Fast Algorithm for Image Deblurring with Total Variation Regularization CAAM technical reports, 2007.
      Z. Wang , A. Bovik , H. Sheikh  and  E. Simoncelli , Image quality assessment: from error visibility to structural similarity, IEEE Transactions on Image Processing, 13 (2004) , 600-612.  doi: 10.1109/TIP.2003.819861.
      O. Whyte , J. Sivic , A. Zisserman  and  J. Ponce , Non-uniform deblurring for shaken images, Int. J. Comput. Vis., 98 (2012) , 168-186.  doi: 10.1007/s11263-011-0502-7.
      C. Wu  and  X. Tai , Augmented lagrangian method, dual methods and split bregman iteration for {ROF} model, in Scale Space and Variational Methods in Computer Vision, (2009) , 502-513. 
      J. Xu , H. Chang  and  J. Qin , Domain decomposition method for image deblurring, Journal of Computational and Applied Mathematics, 271 (2014) , 401-414.  doi: 10.1016/j.cam.2014.03.030.
      L. Xu  and  J. Jia , Two-phase kernel estimation for robust motion deblurring, Computer Vision C ECCV 2010: The series Lecture Notes in Computer Science, 6311 (2010) , 157-170.  doi: 10.1007/978-3-642-15549-9_12.
      L. Xu and J. Jia, Two-phase kernel estimation for robust motion deblurring http://www.cse.cuhk.edu.hk/leojia/projects/robust_deblur/, 2010. doi: 10.1007/978-3-642-15549-9_12.
      X. Zhu , S. Cohen , S. Schiller  and  P. Milanfar , Estimating spatially varying defocus blur from a single image, IEEE Transactions on Image Processing, 22 (2013) , 4879-4891.  doi: 10.1109/TIP.2013.2279316.
      S. Zhuo  and  T. Sim , Defocus map estimation from a single image, Pattern Recognition, 44 (2011) , 1852-1858.  doi: 10.1016/j.patcog.2011.03.009.
  • 加载中




Article Metrics

HTML views(674) PDF downloads(197) Cited by(0)

Access History

Other Articles By Authors



    DownLoad:  Full-Size Img  PowerPoint