\`x^2+y_1+z_12^34\`
Advanced Search
Article Contents
Article Contents

Improving image deblurring

  • *Corresponding author: Mauro Luiz Brandão Junior

    *Corresponding author: Mauro Luiz Brandão Junior 
Abstract / Introduction Full Text(HTML) Figure(10) / Table(5) Related Papers Cited by
  • In this paper, we present a new model to improve image deblurring for the Helsinki Deblur Challenge, promoted by the Finnish Inverse Problems Society in the year of 2021. The challenge consisted in deblurring photographs of random strings of text with varying levels of blur caused by misfocusing the camera. This problem is usually referred to in the literature as out-of-focus deblur. A set of blurred and sharp images was available and also images of dots and other technical targets (horizontal and vertical lines) with the same camera settings. From the observation that the convolution of the sharp images with a uniform disk, which is commonly used as the point spread function (PSF) for the out-of-focus deblur problem, resulted in a image different from the observed blurred images, we observed a pattern that could be modeled as a contrast map. By multiplying this map to the observed images it was possible to significantly improve the image deblurring algorithms, specially for high levels of blur. This map was obtained from the blurred and sharp images that were available. Also, we propose a new deblurring function to be used with the fixed point Regularization by Denoise (RED) algorithm and the results were compared with the Least Absolute Shrinkage and Selection Operator (LASSO) algorithm.

    Mathematics Subject Classification: Primary: 94A08, 68U10.

    Citation:

    \begin{equation} \\ \end{equation}
  • 加载中
  • Figure 1.  In this figure, we illustrate the result of convolving the sharp image with a uniform PSF, with a radius equal to 200 pixels, and compare it with the observed blurred image. Perhaps one may think that the difference is due to the PSF, this PSF was the one that attained the best results when deblurring the image, when compared with other PSFs with different radius and also with the gaussian PSF, which is also used for out-of-focus deblurring. The images correspond to the first sample of the training dataset from the last step (step 19) with the font Verdana

    Figure 2.  In this figure, we compare the observed image with different gamma correction and the ideal blurred image

    Figure 3.  Image deblurring of the observed image and the gamma corrected image. The dark boxes indicate some letters that were improved with gamma correction. Box 1 corresponds to letters "y", box 2 "Qn", box 3 "RG" and box 4 "k"

    Figure 4.  Image deblurring of the observed image and the observed image multiplied by the contrast map. The dark boxes indicate some letters that were greatly improved. Box 1 corresponds to letters "Qy", box 2 "Qn", box 3 "RG" and box 4 "k"

    Figure 5.  Contrast map obtained with pointwise division between the ideal blurred image and the observed blurred image, in step $ 19 $. The deal blurred images were generated by convolving the uniform PSF with the sharp images. All the contrast map were clipped to their median value for better visualization. All maps were summed, clipped to its median value and normalized in order to produce the contrast map that was used in step $ 19 $

    Figure 6.  Processing flowchart illustrating all the steps of the proposed image deblurring scheme

    Figure 7.  Some samples of the deblurred images for step 04 of the test dataset

    Figure 8.  Some samples of the deblurred images for step 09 of the test dataset

    Figure 9.  Some samples of the deblurred images for step 14 of the test dataset

    Figure 10.  Some samples of the deblurred images for step 19 of the test dataset

    Table 1.  Description of the results, specifying the algorithm used, if the contrast map (CM) was used, and if any of the post-processing steps were performed, Median Filtering (MF) and Histogram Equalization (HE)

    Name Deblurring CM MF HE
    LASSO-0 LASSO No No No
    LASSO-1 LASSO No Yes No
    LASSO-2 LASSO No Yes Yes
    LASSO-m0 LASSO Yes No No
    LASSO-m1 LASSO Yes Yes No
    LASSO-m2 LASSO Yes Yes Yes
    RED-0 RED No No No
    RED-1 RED No Yes No
    RED-2 RED No Yes Yes
    RED-m0 RED Yes No No
    RED-m1 RED Yes Yes No
    RED-m2 RED Yes Yes Yes
     | Show Table
    DownLoad: CSV

    Table 2.  Parameters used for generating the PSF, the LASSO and RED algorithms for each step

    Step $ 0 $ $ 1 $ $ 2 $ $ 3 $ $ 4 $ $ 5 $ $ 6 $ $ 7 $ $ 8 $ $ 9 $
    Radius $ 1 $ $ 4 $ $ 12 $ $ 18 $ $ 25 $ $ 37 $ $ 43 $ $ 49 $ $ 61 $ $ 66 $
    LASSO $ \rho $ $ 1.50 $ $ 1.50 $ $ 1.50 $ $ 1.50 $ $ 0.50 $ $ 0.25 $ $ 0.10 $ $ 0.10 $ $ 0.10 $ $ 0.03 $
    $ \lambda $ $ 0.12 $ $ 0.12 $ $ 0.12 $ $ 0.12 $ $ 0.12 $ $ 0.08 $ $ 0.08 $ $ 0.06 $ $ 0.03 $ $ 0.03 $
    RED $ \sigma $ $ 0.1 $ $ 0.1 $ $ 1.0 $ $ 1.0 $ $ 0.5 $ $ 0.5 $ $ 0.25 $ $ 0.25 $ $ 0.25 $ $ 0.15 $
    $ \lambda $ $ 2.0 $ $ 2.0 $ $ 4.0 $ $ 4.0 $ $ 4.0 $ $ 4.0 $ $ 4.00 $ $ 4.00 $ $ 4.00 $ $ 6.00 $
    Step $ 10 $ $ 11 $ $ 12 $ $ 13 $ $ 14 $ $ 15 $ $ 16 $ $ 17 $ $ 18 $ $ 19 $
    Radius $ 74 $ $ 85 $ $ 90 $ $ 96 $ $ 101 $ $ 107 $ $ 140 $ $ 160 $ $ 185 $ $ 200 $
    LASSO $ \rho $ $ 0.03 $ $ 0.03 $ $ 0.03 $ $ 0.02 $ $ 0.02 $ $ 0.02 $ $ 15 $ $ 15 $ $ 10 $ $ 10 $
    $ \lambda $ $ 0.03 $ $ 0.03 $ $ 0.03 $ $ 0.02 $ $ 0.02 $ $ 0.02 $ $ 5 $ $ 5 $ $ 5 $ $ 5 $
    RED $ \sigma $ $ 0.15 $ $ 0.15 $ $ 0.10 $ $ 0.10 $ $ 0.10 $ $ 0.10 $ $ 0.08 $ $ 0.06 $ $ 0.04 $ $ 0.04 $
    $ \lambda $ $ 6.00 $ $ 6.00 $ $ 6.00 $ $ 6.00 $ $ 6.00 $ $ 6.00 $ $ 6.00 $ $ 6.00 $ $ 6.00 $ $ 6.00 $
     | Show Table
    DownLoad: CSV

    Table 3.  OCR scores $ (\%) $ obtained with LASSO. The highest one in each step is highlighted

    Step $ 0 $ $ 1 $ $ 2 $ $ 3 $ $ 4 $ $ 5 $ $ 6 $ $ 7 $ $ 8 $ $ 9 $
    LASSO-0 $ 93.88 $ $ 92.55 $ $ 83.85 $ $ 73.25 $ $ 3.35 $ $ 0.00 $ $ 0.00 $ $ 0.00 $ $ 0.00 $ $ 0.00 $
    LASSO-1 $ 94.80 $ $ 91.28 $ $ 93.05 $ $ 0.00 $ $ 58.05 $ $ 80.93 $ $ 83.80 $ $ 73.58 $ $ 51.80 $ $ 41.28 $
    LASSO-2 $ 94.88 $ $ 91.78 $ $ \bf{95.60} $ $ 2.50 $ $ 20.33 $ $ 54.58 $ $ 15.33 $ $ 12.75 $ $ 0.28 $ $ 1.65 $
    LASSO-m0 $ 94.18 $ $ \bf{92.80} $ $ 88.38 $ $ 80.08 $ $ 67.88 $ $ 71.40 $ $ 60.30 $ $ 61.63 $ $ 56.30 $ $ 13.35 $
    LASSO-m1 $ 92.73 $ $ 91.60 $ $ 93.48 $ $ 90.63 $ $ 89.25 $ $ \bf{94.48} $ $ 81.78 $ $ 71.90 $ $ 80.13 $ $ 70.53 $
    LASSO-m2 $ \bf{95.00} $ $ 92.30 $ $ 94.20 $ $ \bf{93.40} $ $ \bf{90.90} $ $ 93.73 $ $ \bf{85.48} $ $ \bf{77.50} $ $ \bf{84.98} $ $ \bf{72.78} $
    Step $ 10 $ $ 11 $ $ 12 $ $ 13 $ $ 14 $ $ 15 $ $ 16 $ $ 17 $ $ 18 $ $ 19 $
    LASSO-0 $ 0.00 $ $ 0.00 $ $ 0.00 $ $ 0.00 $ $ 0.00 $ $ 0.00 $ $ 0.00 $ $ 0.00 $ $ 0.00 $ $ 0.00 $
    LASSO-1 $ 23.83 $ $ 18.03 $ $ 5.38 $ $ 8.70 $ $ 0.30 $ $ 0.00 $ $ 0.30 $ $ 0.00 $ $ 0.00 $ $ 0.00 $
    LASSO-2 $ 0.00 $ $ 0.30 $ $ 0.95 $ $ 0.00 $ $ 1.43 $ $ 0.65 $ $ 1.33 $ $ 1.48 $ $ 0.43 $ $ 0.00 $
    LASSO-m0 $ 16.78 $ $ 54.20 $ $ 53.78 $ $ 26.20 $ $ 17.88 $ $ 10.80 $ $ 4.70 $ $ 0.00 $ $ 0.00 $ $ 0.75 $
    LASSO-m1 $ \bf{53.30} $ $ \bf{69.10} $ $ 57.10 $ $ 48.95 $ $ 34.03 $ $ 36.80 $ $ 29.28 $ $ 17.78 $ $ 14.03 $ $ 6.78 $
    LASSO-m2 $ 51.43 $ $ 64.50 $ $ \bf{59.25} $ $ \bf{58.88} $ $ \bf{44.00} $ $ \bf{55.70} $ $ \bf{32.50} $ $ \bf{28.93} $ $ \bf{28.30} $ $ \bf{27.35} $
     | Show Table
    DownLoad: CSV

    Table 4.  OCR scores $ (\%) $ obtained with RED. The highest one in each step is highlighted

    Step $ 0 $ $ 1 $ $ 2 $ $ 3 $ $ 4 $ $ 5 $ $ 6 $ $ 7 $ $ 8 $ $ 9 $
    RED-0 $ 94.78 $ $ 0.00 $ $ 82.80 $ $ 71.93 $ $ 1.75 $ $ 16.83 $ $ 0.00 $ $ 0.00 $ $ 0.00 $ $ 0.00 $
    RED-1 $ 95.45 $ $ 95.35 $ $ 93.53 $ $ 31.45 $ $ 9.18 $ $ 60.00 $ $ 12.95 $ $ 7.90 $ $ 0.33 $ $ 0.00 $
    RED-2 $ \bf{96.03} $ $ \bf{95.78} $ $ 93.30 $ $ 0.00 $ $ 28.55 $ $ 52.08 $ $ 24.33 $ $ 14.53 $ $ 1.28 $ $ 1.03 $
    RED-m0 $ 94.10 $ $ 0.00 $ $ 86.55 $ $ 77.65 $ $ 61.03 $ $ 69.33 $ $ 49.88 $ $ 53.78 $ $ 62.05 $ $ 51.03 $
    RED-m1 $ 93.73 $ $ 93.38 $ $ 94.68 $ $ 92.13 $ $ 86.58 $ $ \bf{92.90} $ $ 78.50 $ $ 77.25 $ $ 77.20 $ $ 73.13 $
    RED-m2 $ 94.95 $ $ 95.58 $ $ \bf{95.68} $ $ \bf{94.10} $ $ \bf{91.05} $ $ 92.80 $ $ \bf{84.28} $ $ \bf{83.25} $ $ \bf{85.65} $ $ \bf{82.50} $
    Step $ 10 $ $ 11 $ $ 12 $ $ 13 $ $ 14 $ $ 15 $ $ 16 $ $ 17 $ $ 18 $ $ 19 $
    RED-0 $ 0.00 $ $ 0.00 $ $ 0.00 $ $ 0.00 $ $ 0.00 $ $ 0.00 $ $ 0.00 $ $ 0.00 $ $ 0.00 $ $ 0.00 $
    RED-1 $ 0.00 $ $ 0.00 $ $ 0.00 $ $ 0.45 $ $ 0.00 $ $ 0.00 $ $ 0.00 $ $ 0.00 $ $ 0.00 $ $ 0.00 $
    RED-2 $ 0.35 $ $ 1.08 $ $ 1.23 $ $ 0.85 $ $ 0.00 $ $ 1.13 $ $ 0.00 $ $ 0.00 $ $ 0.00 $ $ 0.00 $
    RED-m0 $ 44.15 $ $ 55.45 $ $ 52.93 $ $ 48.13 $ $ 34.73 $ $ 23.33 $ $ 19.40 $ $ 0.00 $ $ 0.28 $ $ 0.00 $
    RED-m1 $ \bf{62.63} $ $ \bf{71.43} $ $ \bf{68.33} $ $ \bf{68.48} $ $ \bf{57.25} $ $ \bf{67.20} $ $ \bf{53.25} $ $ 37.03 $ $ 26.90 $ $ 21.98 $
    RED-m2 $ 59.48 $ $ 68.05 $ $ 58.00 $ $ 60.23 $ $ 45.10 $ $ 56.03 $ $ 40.23 $ $ \bf{37.25} $ $ \bf{27.80} $ $ \bf{22.30} $
     | Show Table
    DownLoad: CSV

    Table 5.  Average OCR scores $ (\%) $ in all steps, considering LASSO and RED. The highest ones are highlighted

    LASSO RED
    0 $ \bf{17.34} $ $ 13.40 $
    1 $ \bf{36.25} $ $ 20.33 $
    2 $ 19.81 $ $ \bf{20.58} $
    m0 $ 43.57 $ $ \bf{44.19} $
    m1 $ 61.18 $ $ \bf{69.70} $
    m2 $ 66.55 $ $ \bf{68.71} $
     | Show Table
    DownLoad: CSV
  • [1] J. Anger, G. Facciolo and M. Delbracio, Modeling realistic degradations in non-blind deconvolution, In 2018 25th IEEE International Conference on Image Processing (ICIP), (2018), 978-982. doi: 10.1109/ICIP.2018.8451115.
    [2] J. M. Bioucas-Dias, M. A. T. Figueiredo and J. P. Oliveira, Total variation-based image deconvolution: A majorization-minimization approach, In 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings, 2 (2006), Ⅱ-Ⅱ.
    [3] C. A. Bouman, Model based image processing, Purdue University, 341 (2013).
    [4] K. Chen, Introduction to variational image-processing models and applications, Int. J. Comput. Math., 90 (2013), 1-8.  doi: 10.1080/00207160.2012.757073.
    [5] L. Denis, E. Thiébaut and F. Soulez, Fast model of space-variant blurring and its application to deconvolution in astronomy, In 2011 18th IEEE International Conference on Image Processing, (2011), 2817-2820. doi: 10.1109/ICIP.2011.6116257.
    [6] S. FengO. YilmazY. Chen and G. T. Schuster, Zero-offset sections with a deblurring filter in the time domain, GEOPHYSICS, 84 (2019), 239-249. 
    [7] R. Glowinski and A. Marroco, Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de Dirichlet non linéaires, Rev. Française Automat. Informat. Recherche Opérationnelle Sér. Rouge Anal. Numér., 9 (1975), 41-76.  doi: 10.1051/m2an/197509R200411.
    [8] T. GoldsteinB. O'DonoghueS. Setzer and R. Baraniuk, Fast alternating direction optimization methods, SIAM J. Imaging Sci., 7 (2014), 1588-1623.  doi: 10.1137/120896219.
    [9] M. Juvonen, S. Siltanen and F. S. de Moura, Helsinki deblur challenge 2021: Description of photographic data, arXiv preprint, arXiv: 2105.10233, 2021.
    [10] D. Kundur and D. Hatzinakos, Blind image deconvolution, IEEE Signal Processing Magazine, 13 (1996), 43-64.  doi: 10.1109/79.489268.
    [11] S. Lee and S. Cho, Recent Advances in Image Deblurring, SIGGRAPH Asia 2013 Courses, SA '13, New York, NY, USA, 2013, Association for Computing Machinery. doi: 10.1145/2542266.2542272.
    [12] M. E. Moghaddam, A mathematical model to estimate out of focus blur, In 2007 5th International Symposium on Image and Signal Processing and Analysis, (2007), 278-2817. doi: 10.1109/ISPA.2007.4383705.
    [13] S. K. Nayar and M. Ben-Ezra, Motion-based motion deblurring, IEEE Transactions on Pattern Analysis and Machine Intelligence, 26 (2004), 689-698.  doi: 10.1109/TPAMI.2004.1.
    [14] A. Ramdas and R. J. Tibshirani, Fast and flexible admm algorithms for trend filtering, J. Comput. Graph. Statist., 25 (2016), 839-858.  doi: 10.1080/10618600.2015.1054033.
    [15] Y. RomanoM. Elad and P. Milanfar, The little engine that could: Regularization by denoising (red), SIAM Journal on Imaging Sciences, 10 (2017), 1804-1844.  doi: 10.1137/16M1102884.
    [16] L. I. RudinS. Osher and E. Fatemi, Nonlinear total variation based noise removal algorithms, Phys. D, 60 (1992), 259-268.  doi: 10.1016/0167-2789(92)90242-F.
    [17] R. Tibshirani, Regression shrinkage and selection via the LASSO, J. Roy. Statist. Soc. Ser. B, 58 (1996), 267-288.  doi: 10.1111/j.2517-6161.1996.tb02080.x.
    [18] X.-L. ZhaoF. WangT.-Z. HuangM. K. Ng and R. J. Plemmons, Deblurring and sparse unmixing for hyperspectral images, IEEE Transactions on Geoscience and Remote Sensing, 51 (2013), 4045-4058.  doi: 10.1109/TGRS.2012.2227764.
  • 加载中

Figures(10)

Tables(5)

SHARE

Article Metrics

HTML views(2645) PDF downloads(285) Cited by(0)

Access History

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return