Video stabilization of atmospheric turbulence distortion

Yifei Lou; Sung Ha Kang; Stefano Soatto; Andrea L. Bertozzi

doi:10.3934/ipi.2013.7.839

Article Contents

2013, Volume 7, Issue 3: 839-861. Doi: 10.3934/ipi.2013.7.839 open access

This issue Previous Article How to explore the patch space Next Article A conformal approach for surface inpainting

Video stabilization of atmospheric turbulence distortion

1.
Department of Mathematics, University of California Los Angeles, Los Angeles, CA, 90095
2.
School of Mathematics, Georgia Institute of Technology, Atlanta, GA 30332-0160
3.
Computer Science Department, University of California Los Angeles, Los Angeles, CA, 90095

Received: April 2012

Revised: March 2013

Published: August 2013

Abstract / Introduction Related Papers Cited by

Abstract

We present a method to enhance the quality of a video sequence captured through a turbulent atmospheric medium, and give an estimate of the radiance of the distant scene, represented as a ``latent image,'' which is assumed to be static throughout the video. Due to atmospheric turbulence, temporal averaging produces a blurred version of the scene's radiance. We propose a method combining Sobolev gradient and Laplacian to stabilize the video sequence, and a latent image is further found utilizing the ``lucky region" method. The video sequence is stabilized while keeping sharp details, and the latent image shows more consistent straight edges. We analyze the well-posedness for the stabilizing PDE and the linear stability of the numerical scheme.

Keywords:

Mathematics Subject Classification: Primary: 68U10; Secondary: 35B65, 35B35.

Citation:

Related Papers

Cited by

References

[1]	L. Alvarez and L. Mazorra, Signal and image restoration using shock filters and anisotropic diffusion, SIAM Journal on Numerical Analysis, 31 (1994), 590-065.doi: 10.1137/0731032.
[2]	M. Aubailly, M. A. Vorontsov, G. W. Carhat and M. T. Valley, Automated video enhancement from a stream of atmospherically-distorted images: The lucky-region fusion approach, In "Proceedings of SPIE," volume 7463, (2009).doi: 10.1117/12.828332.
[3]	A. Buades, B. Coll and J. M. Morel, A review of image denoising algorithms, with a new one, Multiscale Modeling and Simulation, 4 (2005), 490-530.doi: 10.1137/040616024.
[4]	A. Buades, B. Coll and J.M. Morel, Nonlocal image and movie denoising, International Journal of Computer Vision, 76 (2008), 123-139.
[5]	K. Buskila, S. Towito, E. Shmuel, R. Levi, N. Kopeika, K. Krapels, R. Driggers, R. Vollmerhausen and C. Halford, Atmospheric modulation transfer function in the infrared, Applied Optics, 43 (2004), 471-482.doi: 10.1364/AO.43.000471.
[6]	J. Calder, A. Mansouri and A. Yezzi, Image sharpening via sobolev gradient flows, SIAM Journal on Imaging Sciences, 3 (2010), 981-1014.doi: 10.1137/090771260.
[7]	J. Caviedes and S. Gurbuz, No-reference sharpness metric based on local edge kurtosis, In "IEEE Int. Conf. on Image Processing," 3 (2002), 53-56.doi: 10.1109/ICIP.2002.1038901.
[8]	L. C. Evans, "Partial Differential Equations," Graduate Studies in Mathematics, 19. American Mathematical Society, Providence, RI, 1998.
[9]	D. Frakes, J. Monaco and M. Smith, Suppression of atmospheric turbulence in video using an adaptive control grid interpolation approach, In "IEEE Int. Conf. on Acoustics, Speech and Signal processing (ICASSP)," 3 (2001), 1881-1884.doi: 10.1109/ICASSP.2001.941311.
[10]	D. L. Fried, Optical resolution through a randomly inhomogeneous medium for very long and very short exposures, Journal of the Optical Society of America, 56 (1966), 1372-1379.doi: 10.1364/JOSA.56.001372.
[11]	S. Gepshtein, A. Shtainman, B. Fishbain and L. P. Yaroslavsky, Restoration of atmospheric turbulent video containing real motion using rank filtering and elastic image registration, In "Proceeding of the Eusipco," 2004.
[12]	J. Gilles and S. Osher, "Fried Deconvolution," UCLA CAM Report 11-62, 2011.doi: 10.1117/12.917234.
[13]	P. Hartman, "Ordinary Differential Equations," Corrected reprint. S. M. Hartman, Baltimore, Md., 1973.
[14]	M. Hirsch, S. Sra, B. Scholkopf and S. Harmeling, Efficient filter flow for space-variant multiframe blind deconvolution, IEEE Computer Vision and Pattern Recognition (CVPR), pages 607-614, (2010).doi: 10.1109/CVPR.2010.5540158.
[15]	R. E. Hufnagel and N. R. Stanley, Modulation transfer function associated with image transmission through turbulence media, Journal of the Optical Society of America, 54 (1964), 52-60.doi: 10.1364/JOSA.54.000052.
[16]	J.Gilles, T. Dagobert and C. De Franchis, Atmospheric turbulence restoration by diffeomorphism image registration and blind deconvolution, In "Advanced Concepts for Intelligent Vision Systems" (ACIVS), Oct 2008.
[17]	D. Li, R. Mersereau and S. Simske, Atmospheric turbulence degraded image restoration using principal components analysis, IEEE Geoscience and Remote Sensing Letters, 4 (2007), 340-344.doi: 10.1109/LGRS.2007.895691.
[18]	D. Li and S. Simske, Atmospheric turbulence degraded-image restoration by kurtosis minimization, IEEE Geoscience and Remote Sensing Letters, 6 (2009), 244-247.
[19]	Y. Mao and J. Gilles, Non rigid geometric distortions correction - application to atmospheric turbulence stabilization, Inverse Problems and Imaging, 6 (2012), 531-546.doi: 10.3934/ipi.2012.6.531.
[20]	A. Marquina, Nonlinear inverse scale space methods for total variation blind deconvolution, SIAM Journal on Imaging Sciences, 2 (2009), 64-83.doi: 10.1137/080724289.
[21]	M. Micheli, Y. Lou, S. Soatto and Andrea L. Bertozzi, A linear systems approach to imaging through turbulence, Journal of Mathematical Imaging and Vision July 2013.doi: 10.1007/s10851-012-0410-7.
[22]	A. V. Oppenheim and R. W. Schafer, "Discrete-Time Signal Processing," 2nd Edition Prentice Hall, Upper Saddle River, NJ, 1999.
[23]	P. Perona and J. Malik, Scale-space and edge detection using anisotropic diffusion, IEEE Trans. on Pattern Analysis and Machine Intelligence, 12 (1990), 629-639.doi: 10.1109/34.56205.
[24]	J. Ricklin and F. Davidson, Atmospheric turbulence effects on a partially coherent gaussian beam: implications for free-space laser communication, Journal of the Optical Society of America A, pages 1794-1802, (2002).doi: 10.1364/JOSAA.19.001794.
[25]	M. Roggemann and B. Welsh, "Imaging Through Turbulence," CRC Press, Boca Raton, FL, 1996.doi: 10.1117/1.601043.
[26]	M. Shimizu, S. Yoshimura, M. Tanaka and M. Okutomi, Super-resolution from image sequence under influence of hot-air optical turbulence, In "IEEE Conference on Computer Vision and Pattern Recognition" (CVPR), pages 1-8, (2008).
[27]	G. Sundaramoorthi, A. Yezzi and A. C. Mennucci, Sobolev active contours, International Journal of Computer Vision, 73 (2007), 345-366.
[28]	D. Tofsted, Reanalysis of turbulence effects on short-exposure passive imaging, Optical Engineering, 50 (2011), 016001.doi: 10.1117/1.3532999.
[29]	M. A. Vorontsov and G. W. Carhart, Anisoplanatic imaging through turbulent media: Image recovery by local information fusion from a set of short-exposure images, Journal of the Optical Society of America A, 18 (2001), 1312-1324.doi: 10.1364/JOSAA.18.001312.
[30]	P. Zhang, W. Gong, X. Shen and S. Han, Correlated imaging through atmospheric turbulence, Physical Review A, 82 (2010), 033817.doi: 10.1103/PhysRevA.82.033817.
[31]	X. Zhu and P. Milanfar, Image reconstruction from videos distorted by atmospheric turbulence, In "SPIE Electronic Imaging, Conference on Visual Information Processing and Communication," 2010.doi: 10.1117/12.840127.
[32]	X. Zhu and P. Milanfar, Stabilizing and deblurring atmospheric turbulence, In "IEEE Int. Conf. on Computational Photography (ICCP)," pages 1-8, (2011).doi: 10.1109/ICCPHOT.2011.5753122.
[33]	X. Zhu and P. Milanfar, Removing atmospheric turbulence via space-invariant deconvolution, IEEE Trans. on Pattern Analysis and Machine Intelligence, 35 (2012), 157-170.