Video stabilization of atmospheric turbulence distortion

Previous Article
How to explore the patch space
IPI Home
This Issue
Next Article
A conformal approach for surface inpainting

August 2013, 7(3): 839-861. doi: 10.3934/ipi.2013.7.839

Video stabilization of atmospheric turbulence distortion

Yifei Lou ^1, , Sung Ha Kang ^2, , Stefano Soatto ^3, and Andrea L. Bertozzi ^1,

1.	Department of Mathematics, University of California Los Angeles, Los Angeles, CA, 90095, United States, United States
2.	School of Mathematics, Georgia Institute of Technology, Atlanta, GA 30332-0160
3.	Computer Science Department, University of California Los Angeles, Los Angeles, CA, 90095, United States

Received April 2012 Revised March 2013 Published September 2013

Abstract
Related Papers

We present a method to enhance the quality of a video sequence captured through a turbulent atmospheric medium, and give an estimate of the radiance of the distant scene, represented as a ``latent image,'' which is assumed to be static throughout the video. Due to atmospheric turbulence, temporal averaging produces a blurred version of the scene's radiance. We propose a method combining Sobolev gradient and Laplacian to stabilize the video sequence, and a latent image is further found utilizing the ``lucky region" method. The video sequence is stabilized while keeping sharp details, and the latent image shows more consistent straight edges. We analyze the well-posedness for the stabilizing PDE and the linear stability of the numerical scheme.

Keywords: spectral methods, Imaging through turbulence, anisotropic diffusion, Sobolev gradient sharpening, image fusion..

Mathematics Subject Classification: Primary: 68U10; Secondary: 35B65, 35B3.

Citation: Yifei Lou, Sung Ha Kang, Stefano Soatto, Andrea L. Bertozzi. Video stabilization of atmospheric turbulence distortion. Inverse Problems & Imaging, 2013, 7 (3) : 839-861. doi: 10.3934/ipi.2013.7.839

References:

[1]	L. Alvarez and L. Mazorra, Signal and image restoration using shock filters and anisotropic diffusion,, SIAM Journal on Numerical Analysis, 31 (1994), 590. doi: 10.1137/0731032. Google Scholar
[2]	M. Aubailly, M. A. Vorontsov, G. W. Carhat and M. T. Valley, Automated video enhancement from a stream of atmospherically-distorted images: The lucky-region fusion approach,, In, (2009). doi: 10.1117/12.828332. Google Scholar
[3]	A. Buades, B. Coll and J. M. Morel, A review of image denoising algorithms, with a new one,, Multiscale Modeling and Simulation, 4 (2005), 490. doi: 10.1137/040616024. Google Scholar
[4]	A. Buades, B. Coll and J.M. Morel, Nonlocal image and movie denoising,, International Journal of Computer Vision, 76 (2008), 123. Google Scholar
[5]	K. Buskila, S. Towito, E. Shmuel, R. Levi, N. Kopeika, K. Krapels, R. Driggers, R. Vollmerhausen and C. Halford, Atmospheric modulation transfer function in the infrared,, Applied Optics, 43 (2004), 471. doi: 10.1364/AO.43.000471. Google Scholar
[6]	J. Calder, A. Mansouri and A. Yezzi, Image sharpening via sobolev gradient flows,, SIAM Journal on Imaging Sciences, 3 (2010), 981. doi: 10.1137/090771260. Google Scholar
[7]	J. Caviedes and S. Gurbuz, No-reference sharpness metric based on local edge kurtosis,, In, 3 (2002), 53. doi: 10.1109/ICIP.2002.1038901. Google Scholar
[8]	L. C. Evans, "Partial Differential Equations,", Graduate Studies in Mathematics, (1998). Google Scholar
[9]	D. Frakes, J. Monaco and M. Smith, Suppression of atmospheric turbulence in video using an adaptive control grid interpolation approach,, In, 3 (2001), 1881. doi: 10.1109/ICASSP.2001.941311. Google Scholar
[10]	D. L. Fried, Optical resolution through a randomly inhomogeneous medium for very long and very short exposures,, Journal of the Optical Society of America, 56 (1966), 1372. doi: 10.1364/JOSA.56.001372. Google Scholar
[11]	S. Gepshtein, A. Shtainman, B. Fishbain and L. P. Yaroslavsky, Restoration of atmospheric turbulent video containing real motion using rank filtering and elastic image registration,, In, (2004). Google Scholar
[12]	J. Gilles and S. Osher, "Fried Deconvolution,", UCLA CAM Report 11-62, (2011), 11. doi: 10.1117/12.917234. Google Scholar
[13]	P. Hartman, "Ordinary Differential Equations,", Corrected reprint. S. M. Hartman, (1973). Google Scholar
[14]	M. Hirsch, S. Sra, B. Scholkopf and S. Harmeling, Efficient filter flow for space-variant multiframe blind deconvolution,, IEEE Computer Vision and Pattern Recognition (CVPR), (2010), 607. doi: 10.1109/CVPR.2010.5540158. Google Scholar
[15]	R. E. Hufnagel and N. R. Stanley, Modulation transfer function associated with image transmission through turbulence media,, Journal of the Optical Society of America, 54 (1964), 52. doi: 10.1364/JOSA.54.000052. Google Scholar
[16]	J.Gilles, T. Dagobert and C. De Franchis, Atmospheric turbulence restoration by diffeomorphism image registration and blind deconvolution,, In, (2008). Google Scholar
[17]	D. Li, R. Mersereau and S. Simske, Atmospheric turbulence degraded image restoration using principal components analysis,, IEEE Geoscience and Remote Sensing Letters, 4 (2007), 340. doi: 10.1109/LGRS.2007.895691. Google Scholar
[18]	D. Li and S. Simske, Atmospheric turbulence degraded-image restoration by kurtosis minimization,, IEEE Geoscience and Remote Sensing Letters, 6 (2009), 244. Google Scholar
[19]	Y. Mao and J. Gilles, Non rigid geometric distortions correction - application to atmospheric turbulence stabilization,, Inverse Problems and Imaging, 6 (2012), 531. doi: 10.3934/ipi.2012.6.531. Google Scholar
[20]	A. Marquina, Nonlinear inverse scale space methods for total variation blind deconvolution,, SIAM Journal on Imaging Sciences, 2 (2009), 64. doi: 10.1137/080724289. Google Scholar
[21]	M. Micheli, Y. Lou, S. Soatto and Andrea L. Bertozzi, A linear systems approach to imaging through turbulence,, Journal of Mathematical Imaging and Vision July 2013., (2013). doi: 10.1007/s10851-012-0410-7. Google Scholar
[22]	A. V. Oppenheim and R. W. Schafer, "Discrete-Time Signal Processing," 2nd Edition, Prentice Hall, (1999). Google Scholar
[23]	P. Perona and J. Malik, Scale-space and edge detection using anisotropic diffusion,, IEEE Trans. on Pattern Analysis and Machine Intelligence, 12 (1990), 629. doi: 10.1109/34.56205. Google Scholar
[24]	J. Ricklin and F. Davidson, Atmospheric turbulence effects on a partially coherent gaussian beam: implications for free-space laser communication,, Journal of the Optical Society of America A, (2002), 1794. doi: 10.1364/JOSAA.19.001794. Google Scholar
[25]	M. Roggemann and B. Welsh, "Imaging Through Turbulence,", CRC Press, (1996). doi: 10.1117/1.601043. Google Scholar
[26]	M. Shimizu, S. Yoshimura, M. Tanaka and M. Okutomi, Super-resolution from image sequence under influence of hot-air optical turbulence,, In, (2008), 1. Google Scholar
[27]	G. Sundaramoorthi, A. Yezzi and A. C. Mennucci, Sobolev active contours,, International Journal of Computer Vision, 73 (2007), 345. Google Scholar
[28]	D. Tofsted, Reanalysis of turbulence effects on short-exposure passive imaging,, Optical Engineering, 50 (2011). doi: 10.1117/1.3532999. Google Scholar
[29]	M. A. Vorontsov and G. W. Carhart, Anisoplanatic imaging through turbulent media: Image recovery by local information fusion from a set of short-exposure images,, Journal of the Optical Society of America A, 18 (2001), 1312. doi: 10.1364/JOSAA.18.001312. Google Scholar
[30]	P. Zhang, W. Gong, X. Shen and S. Han, Correlated imaging through atmospheric turbulence,, Physical Review A, 82 (2010). doi: 10.1103/PhysRevA.82.033817. Google Scholar
[31]	X. Zhu and P. Milanfar, Image reconstruction from videos distorted by atmospheric turbulence,, In, (2010). doi: 10.1117/12.840127. Google Scholar
[32]	X. Zhu and P. Milanfar, Stabilizing and deblurring atmospheric turbulence,, In, (2011), 1. doi: 10.1109/ICCPHOT.2011.5753122. Google Scholar
[33]	X. Zhu and P. Milanfar, Removing atmospheric turbulence via space-invariant deconvolution,, IEEE Trans. on Pattern Analysis and Machine Intelligence, 35 (2012), 157. Google Scholar

show all references

References:

[1]	L. Alvarez and L. Mazorra, Signal and image restoration using shock filters and anisotropic diffusion,, SIAM Journal on Numerical Analysis, 31 (1994), 590. doi: 10.1137/0731032. Google Scholar
[2]	M. Aubailly, M. A. Vorontsov, G. W. Carhat and M. T. Valley, Automated video enhancement from a stream of atmospherically-distorted images: The lucky-region fusion approach,, In, (2009). doi: 10.1117/12.828332. Google Scholar
[3]	A. Buades, B. Coll and J. M. Morel, A review of image denoising algorithms, with a new one,, Multiscale Modeling and Simulation, 4 (2005), 490. doi: 10.1137/040616024. Google Scholar
[4]	A. Buades, B. Coll and J.M. Morel, Nonlocal image and movie denoising,, International Journal of Computer Vision, 76 (2008), 123. Google Scholar
[5]	K. Buskila, S. Towito, E. Shmuel, R. Levi, N. Kopeika, K. Krapels, R. Driggers, R. Vollmerhausen and C. Halford, Atmospheric modulation transfer function in the infrared,, Applied Optics, 43 (2004), 471. doi: 10.1364/AO.43.000471. Google Scholar
[6]	J. Calder, A. Mansouri and A. Yezzi, Image sharpening via sobolev gradient flows,, SIAM Journal on Imaging Sciences, 3 (2010), 981. doi: 10.1137/090771260. Google Scholar
[7]	J. Caviedes and S. Gurbuz, No-reference sharpness metric based on local edge kurtosis,, In, 3 (2002), 53. doi: 10.1109/ICIP.2002.1038901. Google Scholar
[8]	L. C. Evans, "Partial Differential Equations,", Graduate Studies in Mathematics, (1998). Google Scholar
[9]	D. Frakes, J. Monaco and M. Smith, Suppression of atmospheric turbulence in video using an adaptive control grid interpolation approach,, In, 3 (2001), 1881. doi: 10.1109/ICASSP.2001.941311. Google Scholar
[10]	D. L. Fried, Optical resolution through a randomly inhomogeneous medium for very long and very short exposures,, Journal of the Optical Society of America, 56 (1966), 1372. doi: 10.1364/JOSA.56.001372. Google Scholar
[11]	S. Gepshtein, A. Shtainman, B. Fishbain and L. P. Yaroslavsky, Restoration of atmospheric turbulent video containing real motion using rank filtering and elastic image registration,, In, (2004). Google Scholar
[12]	J. Gilles and S. Osher, "Fried Deconvolution,", UCLA CAM Report 11-62, (2011), 11. doi: 10.1117/12.917234. Google Scholar
[13]	P. Hartman, "Ordinary Differential Equations,", Corrected reprint. S. M. Hartman, (1973). Google Scholar
[14]	M. Hirsch, S. Sra, B. Scholkopf and S. Harmeling, Efficient filter flow for space-variant multiframe blind deconvolution,, IEEE Computer Vision and Pattern Recognition (CVPR), (2010), 607. doi: 10.1109/CVPR.2010.5540158. Google Scholar
[15]	R. E. Hufnagel and N. R. Stanley, Modulation transfer function associated with image transmission through turbulence media,, Journal of the Optical Society of America, 54 (1964), 52. doi: 10.1364/JOSA.54.000052. Google Scholar
[16]	J.Gilles, T. Dagobert and C. De Franchis, Atmospheric turbulence restoration by diffeomorphism image registration and blind deconvolution,, In, (2008). Google Scholar
[17]	D. Li, R. Mersereau and S. Simske, Atmospheric turbulence degraded image restoration using principal components analysis,, IEEE Geoscience and Remote Sensing Letters, 4 (2007), 340. doi: 10.1109/LGRS.2007.895691. Google Scholar
[18]	D. Li and S. Simske, Atmospheric turbulence degraded-image restoration by kurtosis minimization,, IEEE Geoscience and Remote Sensing Letters, 6 (2009), 244. Google Scholar
[19]	Y. Mao and J. Gilles, Non rigid geometric distortions correction - application to atmospheric turbulence stabilization,, Inverse Problems and Imaging, 6 (2012), 531. doi: 10.3934/ipi.2012.6.531. Google Scholar
[20]	A. Marquina, Nonlinear inverse scale space methods for total variation blind deconvolution,, SIAM Journal on Imaging Sciences, 2 (2009), 64. doi: 10.1137/080724289. Google Scholar
[21]	M. Micheli, Y. Lou, S. Soatto and Andrea L. Bertozzi, A linear systems approach to imaging through turbulence,, Journal of Mathematical Imaging and Vision July 2013., (2013). doi: 10.1007/s10851-012-0410-7. Google Scholar
[22]	A. V. Oppenheim and R. W. Schafer, "Discrete-Time Signal Processing," 2nd Edition, Prentice Hall, (1999). Google Scholar
[23]	P. Perona and J. Malik, Scale-space and edge detection using anisotropic diffusion,, IEEE Trans. on Pattern Analysis and Machine Intelligence, 12 (1990), 629. doi: 10.1109/34.56205. Google Scholar
[24]	J. Ricklin and F. Davidson, Atmospheric turbulence effects on a partially coherent gaussian beam: implications for free-space laser communication,, Journal of the Optical Society of America A, (2002), 1794. doi: 10.1364/JOSAA.19.001794. Google Scholar
[25]	M. Roggemann and B. Welsh, "Imaging Through Turbulence,", CRC Press, (1996). doi: 10.1117/1.601043. Google Scholar
[26]	M. Shimizu, S. Yoshimura, M. Tanaka and M. Okutomi, Super-resolution from image sequence under influence of hot-air optical turbulence,, In, (2008), 1. Google Scholar
[27]	G. Sundaramoorthi, A. Yezzi and A. C. Mennucci, Sobolev active contours,, International Journal of Computer Vision, 73 (2007), 345. Google Scholar
[28]	D. Tofsted, Reanalysis of turbulence effects on short-exposure passive imaging,, Optical Engineering, 50 (2011). doi: 10.1117/1.3532999. Google Scholar
[29]	M. A. Vorontsov and G. W. Carhart, Anisoplanatic imaging through turbulent media: Image recovery by local information fusion from a set of short-exposure images,, Journal of the Optical Society of America A, 18 (2001), 1312. doi: 10.1364/JOSAA.18.001312. Google Scholar
[30]	P. Zhang, W. Gong, X. Shen and S. Han, Correlated imaging through atmospheric turbulence,, Physical Review A, 82 (2010). doi: 10.1103/PhysRevA.82.033817. Google Scholar
[31]	X. Zhu and P. Milanfar, Image reconstruction from videos distorted by atmospheric turbulence,, In, (2010). doi: 10.1117/12.840127. Google Scholar
[32]	X. Zhu and P. Milanfar, Stabilizing and deblurring atmospheric turbulence,, In, (2011), 1. doi: 10.1109/ICCPHOT.2011.5753122. Google Scholar
[33]	X. Zhu and P. Milanfar, Removing atmospheric turbulence via space-invariant deconvolution,, IEEE Trans. on Pattern Analysis and Machine Intelligence, 35 (2012), 157. Google Scholar

[1]	Kaitlyn (Voccola) Muller. SAR correlation imaging and anisotropic scattering. Inverse Problems & Imaging, 2018, 12 (3) : 697-731. doi: 10.3934/ipi.2018030
[2]	Hong Seng Sim, Wah June Leong, Chuei Yee Chen, Siti Nur Iqmal Ibrahim. Multi-step spectral gradient methods with modified weak secant relation for large scale unconstrained optimization. Numerical Algebra, Control & Optimization, 2018, 8 (3) : 377-387. doi: 10.3934/naco.2018024
[3]	Zhong Wan, Chaoming Hu, Zhanlu Yang. A spectral PRP conjugate gradient methods for nonconvex optimization problem based on modified line search. Discrete & Continuous Dynamical Systems - B, 2011, 16 (4) : 1157-1169. doi: 10.3934/dcdsb.2011.16.1157
[4]	Jiangchuan Fan, Xinyu Guo, Jianjun Du, Weiliang Wen, Xianju Lu, Brahmani Louiza. Analysis of the clustering fusion algorithm for multi-band color image. Discrete & Continuous Dynamical Systems - S, 2019, 12 (4&5) : 1233-1249. doi: 10.3934/dcdss.2019085
[5]	Moulay Rchid Sidi Ammi, Ismail Jamiai. Finite difference and Legendre spectral method for a time-fractional diffusion-convection equation for image restoration. Discrete & Continuous Dynamical Systems - S, 2018, 11 (1) : 103-117. doi: 10.3934/dcdss.2018007
[6]	Nicolas Lermé, François Malgouyres, Dominique Hamoir, Emmanuelle Thouin. Bayesian image restoration for mosaic active imaging. Inverse Problems & Imaging, 2014, 8 (3) : 733-760. doi: 10.3934/ipi.2014.8.733
[7]	Yuhong Dai, Ya-xiang Yuan. Analysis of monotone gradient methods. Journal of Industrial & Management Optimization, 2005, 1 (2) : 181-192. doi: 10.3934/jimo.2005.1.181
[8]	Shenglong Hu, Zheng-Hai Huang, Hong-Yan Ni, Liqun Qi. Positive definiteness of Diffusion Kurtosis Imaging. Inverse Problems & Imaging, 2012, 6 (1) : 57-75. doi: 10.3934/ipi.2012.6.57
[9]	Antoni Buades, Bartomeu Coll, Jose-Luis Lisani, Catalina Sbert. Conditional image diffusion. Inverse Problems & Imaging, 2007, 1 (4) : 593-608. doi: 10.3934/ipi.2007.1.593
[10]	Jianjun Zhang, Yunyi Hu, James G. Nagy. A scaled gradient method for digital tomographic image reconstruction. Inverse Problems & Imaging, 2018, 12 (1) : 239-259. doi: 10.3934/ipi.2018010
[11]	Timothy Blass, Rafael De La Llave, Enrico Valdinoci. A comparison principle for a Sobolev gradient semi-flow. Communications on Pure & Applied Analysis, 2011, 10 (1) : 69-91. doi: 10.3934/cpaa.2011.10.69
[12]	Wanyou Cheng, Zixin Chen, Donghui Li. Nomonotone spectral gradient method for sparse recovery. Inverse Problems & Imaging, 2015, 9 (3) : 815-833. doi: 10.3934/ipi.2015.9.815
[13]	Jianqing Chen. Best constant of 3D Anisotropic Sobolev inequality and its applications. Communications on Pure & Applied Analysis, 2010, 9 (3) : 655-666. doi: 10.3934/cpaa.2010.9.655
[14]	Robert I. McLachlan, G. R. W. Quispel. Discrete gradient methods have an energy conservation law. Discrete & Continuous Dynamical Systems - A, 2014, 34 (3) : 1099-1104. doi: 10.3934/dcds.2014.34.1099
[15]	Giacomo Frassoldati, Luca Zanni, Gaetano Zanghirati. New adaptive stepsize selections in gradient methods. Journal of Industrial & Management Optimization, 2008, 4 (2) : 299-312. doi: 10.3934/jimo.2008.4.299
[16]	Richard A. Norton, David I. McLaren, G. R. W. Quispel, Ari Stern, Antonella Zanna. Projection methods and discrete gradient methods for preserving first integrals of ODEs. Discrete & Continuous Dynamical Systems - A, 2015, 35 (5) : 2079-2098. doi: 10.3934/dcds.2015.35.2079
[17]	Yunho Kim, Luminita A. Vese. Image recovery using functions of bounded variation and Sobolev spaces of negative differentiability. Inverse Problems & Imaging, 2009, 3 (1) : 43-68. doi: 10.3934/ipi.2009.3.43
[18]	Samuel Amstutz, Antonio André Novotny, Nicolas Van Goethem. Minimal partitions and image classification using a gradient-free perimeter approximation. Inverse Problems & Imaging, 2014, 8 (2) : 361-387. doi: 10.3934/ipi.2014.8.361
[19]	Juan C. Moreno, V. B. Surya Prasath, João C. Neves. Color image processing by vectorial total variation with gradient channels coupling. Inverse Problems & Imaging, 2016, 10 (2) : 461-497. doi: 10.3934/ipi.2016008
[20]	Nam-Yong Lee, Bradley J. Lucier. Preconditioned conjugate gradient method for boundary artifact-free image deblurring. Inverse Problems & Imaging, 2016, 10 (1) : 195-225. doi: 10.3934/ipi.2016.10.195

2018 Impact Factor: 1.469

American Institute of Mathematical Sciences