American Institute of Mathematical Sciences

Advanced
August  2013, 7(3): 839-861. doi: 10.3934/ipi.2013.7.839

Video stabilization of atmospheric turbulence distortion

Yifei Lou 1, , Sung Ha Kang 2, , Stefano Soatto 3, and  Andrea L. Bertozzi 1,

1. 

Department of Mathematics, University of California Los Angeles, Los Angeles, CA, 90095, United States, United States
2. 

School of Mathematics, Georgia Institute of Technology, Atlanta, GA 30332-0160
3. 

Computer Science Department, University of California Los Angeles, Los Angeles, CA, 90095, United States

Received  April 2012 Revised  March 2013 Published  September 2013

We present a method to enhance the quality of a video sequence captured through a turbulent atmospheric medium, and give an estimate of the radiance of the distant scene, represented as a ``latent image,'' which is assumed to be static throughout the video. Due to atmospheric turbulence, temporal averaging produces a blurred version of the scene's radiance. We propose a method combining Sobolev gradient and Laplacian to stabilize the video sequence, and a latent image is further found utilizing the ``lucky region" method. The video sequence is stabilized while keeping sharp details, and the latent image shows more consistent straight edges. We analyze the well-posedness for the stabilizing PDE and the linear stability of the numerical scheme.
Keywords: spectral methods, Imaging through turbulence, anisotropic diffusion, Sobolev gradient sharpening, image fusion..
Mathematics Subject Classification: Primary: 68U10; Secondary: 35B65, 35B3.
Citation: Yifei Lou, Sung Ha Kang, Stefano Soatto, Andrea L. Bertozzi. Video stabilization of atmospheric turbulence distortion. Inverse Problems & Imaging, 2013, 7 (3) : 839-861. doi: 10.3934/ipi.2013.7.839
References:
[1]

L. Alvarez and L. Mazorra, Signal and image restoration using shock filters and anisotropic diffusion,, SIAM Journal on Numerical Analysis, 31 (1994), 590.  doi: 10.1137/0731032.  Google Scholar

[2]

M. Aubailly, M. A. Vorontsov, G. W. Carhat and M. T. Valley, Automated video enhancement from a stream of atmospherically-distorted images: The lucky-region fusion approach,, In, (2009).  doi: 10.1117/12.828332.  Google Scholar

[3]

A. Buades, B. Coll and J. M. Morel, A review of image denoising algorithms, with a new one,, Multiscale Modeling and Simulation, 4 (2005), 490.  doi: 10.1137/040616024.  Google Scholar

[4]

A. Buades, B. Coll and J.M. Morel, Nonlocal image and movie denoising,, International Journal of Computer Vision, 76 (2008), 123.   Google Scholar

[5]

K. Buskila, S. Towito, E. Shmuel, R. Levi, N. Kopeika, K. Krapels, R. Driggers, R. Vollmerhausen and C. Halford, Atmospheric modulation transfer function in the infrared,, Applied Optics, 43 (2004), 471.  doi: 10.1364/AO.43.000471.  Google Scholar

[6]

J. Calder, A. Mansouri and A. Yezzi, Image sharpening via sobolev gradient flows,, SIAM Journal on Imaging Sciences, 3 (2010), 981.  doi: 10.1137/090771260.  Google Scholar

[7]

J. Caviedes and S. Gurbuz, No-reference sharpness metric based on local edge kurtosis,, In, 3 (2002), 53.  doi: 10.1109/ICIP.2002.1038901.  Google Scholar

[8]

L. C. Evans, "Partial Differential Equations,", Graduate Studies in Mathematics, (1998).   Google Scholar

[9]

D. Frakes, J. Monaco and M. Smith, Suppression of atmospheric turbulence in video using an adaptive control grid interpolation approach,, In, 3 (2001), 1881.  doi: 10.1109/ICASSP.2001.941311.  Google Scholar

[10]

D. L. Fried, Optical resolution through a randomly inhomogeneous medium for very long and very short exposures,, Journal of the Optical Society of America, 56 (1966), 1372.  doi: 10.1364/JOSA.56.001372.  Google Scholar

[11]

S. Gepshtein, A. Shtainman, B. Fishbain and L. P. Yaroslavsky, Restoration of atmospheric turbulent video containing real motion using rank filtering and elastic image registration,, In, (2004).   Google Scholar

[12]

J. Gilles and S. Osher, "Fried Deconvolution,", UCLA CAM Report 11-62, (2011), 11.  doi: 10.1117/12.917234.  Google Scholar

[13]

P. Hartman, "Ordinary Differential Equations,", Corrected reprint. S. M. Hartman, (1973).   Google Scholar

[14]

M. Hirsch, S. Sra, B. Scholkopf and S. Harmeling, Efficient filter flow for space-variant multiframe blind deconvolution,, IEEE Computer Vision and Pattern Recognition (CVPR), (2010), 607.  doi: 10.1109/CVPR.2010.5540158.  Google Scholar

[15]

R. E. Hufnagel and N. R. Stanley, Modulation transfer function associated with image transmission through turbulence media,, Journal of the Optical Society of America, 54 (1964), 52.  doi: 10.1364/JOSA.54.000052.  Google Scholar

[16]

J.Gilles, T. Dagobert and C. De Franchis, Atmospheric turbulence restoration by diffeomorphism image registration and blind deconvolution,, In, (2008).   Google Scholar

[17]

D. Li, R. Mersereau and S. Simske, Atmospheric turbulence degraded image restoration using principal components analysis,, IEEE Geoscience and Remote Sensing Letters, 4 (2007), 340.  doi: 10.1109/LGRS.2007.895691.  Google Scholar

[18]

D. Li and S. Simske, Atmospheric turbulence degraded-image restoration by kurtosis minimization,, IEEE Geoscience and Remote Sensing Letters, 6 (2009), 244.   Google Scholar

[19]

Y. Mao and J. Gilles, Non rigid geometric distortions correction - application to atmospheric turbulence stabilization,, Inverse Problems and Imaging, 6 (2012), 531.  doi: 10.3934/ipi.2012.6.531.  Google Scholar

[20]

A. Marquina, Nonlinear inverse scale space methods for total variation blind deconvolution,, SIAM Journal on Imaging Sciences, 2 (2009), 64.  doi: 10.1137/080724289.  Google Scholar

[21]

M. Micheli, Y. Lou, S. Soatto and Andrea L. Bertozzi, A linear systems approach to imaging through turbulence,, Journal of Mathematical Imaging and Vision July 2013., (2013).  doi: 10.1007/s10851-012-0410-7.  Google Scholar

[22]

A. V. Oppenheim and R. W. Schafer, "Discrete-Time Signal Processing," 2nd Edition, Prentice Hall, (1999).   Google Scholar

[23]

P. Perona and J. Malik, Scale-space and edge detection using anisotropic diffusion,, IEEE Trans. on Pattern Analysis and Machine Intelligence, 12 (1990), 629.  doi: 10.1109/34.56205.  Google Scholar

[24]

J. Ricklin and F. Davidson, Atmospheric turbulence effects on a partially coherent gaussian beam: implications for free-space laser communication,, Journal of the Optical Society of America A, (2002), 1794.  doi: 10.1364/JOSAA.19.001794.  Google Scholar

[25]

M. Roggemann and B. Welsh, "Imaging Through Turbulence,", CRC Press, (1996).  doi: 10.1117/1.601043.  Google Scholar

[26]

M. Shimizu, S. Yoshimura, M. Tanaka and M. Okutomi, Super-resolution from image sequence under influence of hot-air optical turbulence,, In, (2008), 1.   Google Scholar

[27]

G. Sundaramoorthi, A. Yezzi and A. C. Mennucci, Sobolev active contours,, International Journal of Computer Vision, 73 (2007), 345.   Google Scholar

[28]

D. Tofsted, Reanalysis of turbulence effects on short-exposure passive imaging,, Optical Engineering, 50 (2011).  doi: 10.1117/1.3532999.  Google Scholar

[29]

M. A. Vorontsov and G. W. Carhart, Anisoplanatic imaging through turbulent media: Image recovery by local information fusion from a set of short-exposure images,, Journal of the Optical Society of America A, 18 (2001), 1312.  doi: 10.1364/JOSAA.18.001312.  Google Scholar

[30]

P. Zhang, W. Gong, X. Shen and S. Han, Correlated imaging through atmospheric turbulence,, Physical Review A, 82 (2010).  doi: 10.1103/PhysRevA.82.033817.  Google Scholar

[31]

X. Zhu and P. Milanfar, Image reconstruction from videos distorted by atmospheric turbulence,, In, (2010).  doi: 10.1117/12.840127.  Google Scholar

[32]

X. Zhu and P. Milanfar, Stabilizing and deblurring atmospheric turbulence,, In, (2011), 1.  doi: 10.1109/ICCPHOT.2011.5753122.  Google Scholar

[33]

X. Zhu and P. Milanfar, Removing atmospheric turbulence via space-invariant deconvolution,, IEEE Trans. on Pattern Analysis and Machine Intelligence, 35 (2012), 157.   Google Scholar

show all references

References:
[1]

L. Alvarez and L. Mazorra, Signal and image restoration using shock filters and anisotropic diffusion,, SIAM Journal on Numerical Analysis, 31 (1994), 590.  doi: 10.1137/0731032.  Google Scholar

[2]

M. Aubailly, M. A. Vorontsov, G. W. Carhat and M. T. Valley, Automated video enhancement from a stream of atmospherically-distorted images: The lucky-region fusion approach,, In, (2009).  doi: 10.1117/12.828332.  Google Scholar

[3]

A. Buades, B. Coll and J. M. Morel, A review of image denoising algorithms, with a new one,, Multiscale Modeling and Simulation, 4 (2005), 490.  doi: 10.1137/040616024.  Google Scholar

[4]

A. Buades, B. Coll and J.M. Morel, Nonlocal image and movie denoising,, International Journal of Computer Vision, 76 (2008), 123.   Google Scholar

[5]

K. Buskila, S. Towito, E. Shmuel, R. Levi, N. Kopeika, K. Krapels, R. Driggers, R. Vollmerhausen and C. Halford, Atmospheric modulation transfer function in the infrared,, Applied Optics, 43 (2004), 471.  doi: 10.1364/AO.43.000471.  Google Scholar

[6]

J. Calder, A. Mansouri and A. Yezzi, Image sharpening via sobolev gradient flows,, SIAM Journal on Imaging Sciences, 3 (2010), 981.  doi: 10.1137/090771260.  Google Scholar

[7]

J. Caviedes and S. Gurbuz, No-reference sharpness metric based on local edge kurtosis,, In, 3 (2002), 53.  doi: 10.1109/ICIP.2002.1038901.  Google Scholar

[8]

L. C. Evans, "Partial Differential Equations,", Graduate Studies in Mathematics, (1998).   Google Scholar

[9]

D. Frakes, J. Monaco and M. Smith, Suppression of atmospheric turbulence in video using an adaptive control grid interpolation approach,, In, 3 (2001), 1881.  doi: 10.1109/ICASSP.2001.941311.  Google Scholar

[10]

D. L. Fried, Optical resolution through a randomly inhomogeneous medium for very long and very short exposures,, Journal of the Optical Society of America, 56 (1966), 1372.  doi: 10.1364/JOSA.56.001372.  Google Scholar

[11]

S. Gepshtein, A. Shtainman, B. Fishbain and L. P. Yaroslavsky, Restoration of atmospheric turbulent video containing real motion using rank filtering and elastic image registration,, In, (2004).   Google Scholar

[12]

J. Gilles and S. Osher, "Fried Deconvolution,", UCLA CAM Report 11-62, (2011), 11.  doi: 10.1117/12.917234.  Google Scholar

[13]

P. Hartman, "Ordinary Differential Equations,", Corrected reprint. S. M. Hartman, (1973).   Google Scholar

[14]

M. Hirsch, S. Sra, B. Scholkopf and S. Harmeling, Efficient filter flow for space-variant multiframe blind deconvolution,, IEEE Computer Vision and Pattern Recognition (CVPR), (2010), 607.  doi: 10.1109/CVPR.2010.5540158.  Google Scholar

[15]

R. E. Hufnagel and N. R. Stanley, Modulation transfer function associated with image transmission through turbulence media,, Journal of the Optical Society of America, 54 (1964), 52.  doi: 10.1364/JOSA.54.000052.  Google Scholar

[16]

J.Gilles, T. Dagobert and C. De Franchis, Atmospheric turbulence restoration by diffeomorphism image registration and blind deconvolution,, In, (2008).   Google Scholar

[17]

D. Li, R. Mersereau and S. Simske, Atmospheric turbulence degraded image restoration using principal components analysis,, IEEE Geoscience and Remote Sensing Letters, 4 (2007), 340.  doi: 10.1109/LGRS.2007.895691.  Google Scholar

[18]

D. Li and S. Simske, Atmospheric turbulence degraded-image restoration by kurtosis minimization,, IEEE Geoscience and Remote Sensing Letters, 6 (2009), 244.   Google Scholar

[19]

Y. Mao and J. Gilles, Non rigid geometric distortions correction - application to atmospheric turbulence stabilization,, Inverse Problems and Imaging, 6 (2012), 531.  doi: 10.3934/ipi.2012.6.531.  Google Scholar

[20]

A. Marquina, Nonlinear inverse scale space methods for total variation blind deconvolution,, SIAM Journal on Imaging Sciences, 2 (2009), 64.  doi: 10.1137/080724289.  Google Scholar

[21]

M. Micheli, Y. Lou, S. Soatto and Andrea L. Bertozzi, A linear systems approach to imaging through turbulence,, Journal of Mathematical Imaging and Vision July 2013., (2013).  doi: 10.1007/s10851-012-0410-7.  Google Scholar

[22]

A. V. Oppenheim and R. W. Schafer, "Discrete-Time Signal Processing," 2nd Edition, Prentice Hall, (1999).   Google Scholar

[23]

P. Perona and J. Malik, Scale-space and edge detection using anisotropic diffusion,, IEEE Trans. on Pattern Analysis and Machine Intelligence, 12 (1990), 629.  doi: 10.1109/34.56205.  Google Scholar

[24]

J. Ricklin and F. Davidson, Atmospheric turbulence effects on a partially coherent gaussian beam: implications for free-space laser communication,, Journal of the Optical Society of America A, (2002), 1794.  doi: 10.1364/JOSAA.19.001794.  Google Scholar

[25]

M. Roggemann and B. Welsh, "Imaging Through Turbulence,", CRC Press, (1996).  doi: 10.1117/1.601043.  Google Scholar

[26]

M. Shimizu, S. Yoshimura, M. Tanaka and M. Okutomi, Super-resolution from image sequence under influence of hot-air optical turbulence,, In, (2008), 1.   Google Scholar

[27]

G. Sundaramoorthi, A. Yezzi and A. C. Mennucci, Sobolev active contours,, International Journal of Computer Vision, 73 (2007), 345.   Google Scholar

[28]

D. Tofsted, Reanalysis of turbulence effects on short-exposure passive imaging,, Optical Engineering, 50 (2011).  doi: 10.1117/1.3532999.  Google Scholar

[29]

M. A. Vorontsov and G. W. Carhart, Anisoplanatic imaging through turbulent media: Image recovery by local information fusion from a set of short-exposure images,, Journal of the Optical Society of America A, 18 (2001), 1312.  doi: 10.1364/JOSAA.18.001312.  Google Scholar

[30]

P. Zhang, W. Gong, X. Shen and S. Han, Correlated imaging through atmospheric turbulence,, Physical Review A, 82 (2010).  doi: 10.1103/PhysRevA.82.033817.  Google Scholar

[31]

X. Zhu and P. Milanfar, Image reconstruction from videos distorted by atmospheric turbulence,, In, (2010).  doi: 10.1117/12.840127.  Google Scholar

[32]

X. Zhu and P. Milanfar, Stabilizing and deblurring atmospheric turbulence,, In, (2011), 1.  doi: 10.1109/ICCPHOT.2011.5753122.  Google Scholar

[33]

X. Zhu and P. Milanfar, Removing atmospheric turbulence via space-invariant deconvolution,, IEEE Trans. on Pattern Analysis and Machine Intelligence, 35 (2012), 157.   Google Scholar

[1]

Predrag S. Stanimirović, Branislav Ivanov, Haifeng Ma, Dijana Mosić. A survey of gradient methods for solving nonlinear optimization. Electronic Research Archive, 2020, 28 (4) : 1573-1624. doi: 10.3934/era.2020115
[2]

Abdelghafour Atlas, Mostafa Bendahmane, Fahd Karami, Driss Meskine, Omar Oubbih. A nonlinear fractional reaction-diffusion system applied to image denoising and decomposition. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020321
[3]

Thomas Frenzel, Matthias Liero. Effective diffusion in thin structures via generalized gradient systems and EDP-convergence. Discrete & Continuous Dynamical Systems - S, 2021, 14 (1) : 395-425. doi: 10.3934/dcdss.2020345
[4]

Peter Frolkovič, Karol Mikula, Jooyoung Hahn, Dirk Martin, Branislav Basara. Flux balanced approximation with least-squares gradient for diffusion equation on polyhedral mesh. Discrete & Continuous Dynamical Systems - S, 2021, 14 (3) : 865-879. doi: 10.3934/dcdss.2020350
[5]

Vincent Ducrot, Pascal Frey, Alexandra Claisse. Levelsets and anisotropic mesh adaptation. Discrete & Continuous Dynamical Systems - A, 2009, 23 (1&2) : 165-183. doi: 10.3934/dcds.2009.23.165
[6]

Beom-Seok Han, Kyeong-Hun Kim, Daehan Park. A weighted Sobolev space theory for the diffusion-wave equations with time-fractional derivatives on $ C^{1} $ domains. Discrete & Continuous Dynamical Systems - A, 2021  doi: 10.3934/dcds.2021002
[7]

Kung-Ching Chang, Xuefeng Wang, Xie Wu. On the spectral theory of positive operators and PDE applications. Discrete & Continuous Dynamical Systems - A, 2020, 40 (6) : 3171-3200. doi: 10.3934/dcds.2020054
[8]

Giuseppina Guatteri, Federica Masiero. Stochastic maximum principle for problems with delay with dependence on the past through general measures. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020048
[9]

Joel Kübler, Tobias Weth. Spectral asymptotics of radial solutions and nonradial bifurcation for the Hénon equation. Discrete & Continuous Dynamical Systems - A, 2020, 40 (6) : 3629-3656. doi: 10.3934/dcds.2020032
[10]

Olivier Pironneau, Alexei Lozinski, Alain Perronnet, Frédéric Hecht. Numerical zoom for multiscale problems with an application to flows through porous media. Discrete & Continuous Dynamical Systems - A, 2009, 23 (1&2) : 265-280. doi: 10.3934/dcds.2009.23.265
[11]

Bopeng Rao, Zhuangyi Liu. A spectral approach to the indirect boundary control of a system of weakly coupled wave equations. Discrete & Continuous Dynamical Systems - A, 2009, 23 (1&2) : 399-414. doi: 10.3934/dcds.2009.23.399
[12]

Mehdi Bastani, Davod Khojasteh Salkuyeh. On the GSOR iteration method for image restoration. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 27-43. doi: 10.3934/naco.2020013
[13]

Yao Nie, Jia Yuan. The Littlewood-Paley $ pth $-order moments in three-dimensional MHD turbulence. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020397
[14]

Josselin Garnier, Knut Sølna. Enhanced Backscattering of a partially coherent field from an anisotropic random lossy medium. Discrete & Continuous Dynamical Systems - B, 2021, 26 (2) : 1171-1195. doi: 10.3934/dcdsb.2020158
[15]

Hongbo Guan, Yong Yang, Huiqing Zhu. A nonuniform anisotropic FEM for elliptic boundary layer optimal control problems. Discrete & Continuous Dynamical Systems - B, 2021, 26 (3) : 1711-1722. doi: 10.3934/dcdsb.2020179
[16]

Andreas Kreuml. The anisotropic fractional isoperimetric problem with respect to unconditional unit balls. Communications on Pure & Applied Analysis, 2021, 20 (2) : 783-799. doi: 10.3934/cpaa.2020290
[17]

Alexander Dabrowski, Ahcene Ghandriche, Mourad Sini. Mathematical analysis of the acoustic imaging modality using bubbles as contrast agents at nearly resonating frequencies. Inverse Problems & Imaging, , () : -. doi: 10.3934/ipi.2021005
[18]

Weihong Guo, Yifei Lou, Jing Qin, Ming Yan. IPI special issue on "mathematical/statistical approaches in data science" in the Inverse Problem and Imaging. Inverse Problems & Imaging, 2021, 15 (1) : I-I. doi: 10.3934/ipi.2021007
[19]

Philippe G. Ciarlet, Liliana Gratie, Cristinel Mardare. Intrinsic methods in elasticity: a mathematical survey. Discrete & Continuous Dynamical Systems - A, 2009, 23 (1&2) : 133-164. doi: 10.3934/dcds.2009.23.133
[20]

Alessandro Carbotti, Giovanni E. Comi. A note on Riemann-Liouville fractional Sobolev spaces. Communications on Pure & Applied Analysis, 2021, 20 (1) : 17-54. doi: 10.3934/cpaa.2020255

2019 Impact Factor: 1.373

Tools

Metrics

  • PDF downloads (164)
  • HTML views (0)
  • Cited by (13)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences