# American Institute of Mathematical Sciences

May  2012, 6(2): 201-214. doi: 10.3934/ipi.2012.6.201

## Surveillance video processing using compressive sensing

 1 Bell Labs, Alcatel-Lucent, 700 Mountain Ave, Murray Hill, NJ 07974, United States 2 Dept. of Computational and Applied Math., Rice University, Houston, TX 77005, United States 3 Dept. of Math., National Univ. of Singapore, 119076, Singapore

Received  December 2011 Revised  February 2012 Published  May 2012

A compressive sensing method combined with decomposition of a matrix formed with image frames of a surveillance video into low rank and sparse matrices is proposed to segment the background and extract moving objects in a surveillance video. The video is acquired by compressive measurements, and the measurements are used to reconstruct the video by a low rank and sparse decomposition of matrix. The low rank component represents the background, and the sparse component is used to identify moving objects in the surveillance video. The decomposition is performed by an augmented Lagrangian alternating direction method. Experiments are carried out to demonstrate that moving objects can be reliably extracted with a small amount of measurements.
Citation: Hong Jiang, Wei Deng, Zuowei Shen. Surveillance video processing using compressive sensing. Inverse Problems & Imaging, 2012, 6 (2) : 201-214. doi: 10.3934/ipi.2012.6.201
##### References:
 [1] Y. Benezeth, P. M. Jodoin, B. Emile, H. Laurent and C. Rosenberger, Comparative study of background subtraction algorithms,, J. Electron. Imaging, 19 (2010).  doi: 10.1117/1.3456695.  Google Scholar [2] J.-F. Cai, E. J. Candès and Z. Shen, A singular value thresholding algorithm for matrix completion,, SIAM Journal on Optimization, 20 (2010), 1956.  doi: 10.1137/080738970.  Google Scholar [3] J.-F. Cai, S. Osher and Z. Shen, Split Bregman methods and frame based image restoration,, Multiscale Modeling and Simulation: A SIAM Interdisciplinary Journal, 8 (): 337.   Google Scholar [4] E.-J. Candès, X. Li, Y. Ma and J. Wright, Robust principal component analysis?,, Journal of ACM, 58 (2011).   Google Scholar [5] E. J. Candès, J. Romberg and T. Tao, Stable signal recovery from incomplete and inaccurate measurements,, Comm. Pure Appl. Math., 59 (2006), 1207.  doi: 10.1002/cpa.20124.  Google Scholar [6] V. Cevher, A. Sankaranarayanan, M. Duarte, D. Reddy, R. Baraniuk and R. Chellappa, Compressive sensing for background subtraction,, Computer Vision-ECCV 2008, (2008), 155.   Google Scholar [7] I. Daubechies, B. Han, A. Ron and Z. Shen, Framelets: MRA-based constructions of wavelet frames,, Applied and Computational Harmonic Analysis, 14 (2003), 1.  doi: 10.1016/S1063-5203(02)00511-0.  Google Scholar [8] W. Deng, W. Yin and Y. Zhang, "Group Sparse Optimization by Alternating Direction Method,", TR11-06, (2011), 11.   Google Scholar [9] B. Dong and Z. Shen, "MRA-Based Wavelet Frames and Applications,", IAS Lecture Notes Series, (2010).   Google Scholar [10] Y. Dong, G. N. DeSouza and T. X. Han, Illumination invariant foreground detection using multi-subspace learning,, International Journal of Knowledge-based and Intelligent Engineering Systems, 14 (2010), 31.   Google Scholar [11] D. Donoho, Compressed sensing,, IEEE Trans. on Information Theory, 52 (2006), 1289.  doi: 10.1109/TIT.2006.871582.  Google Scholar [12] M. Fornasier and H. Rauhut, Recovery algorithms for vector-valued data with joint sparsity constraints,, SIAM J. Numer. Anal., 46 (2008), 577.  doi: 10.1137/0606668909.  Google Scholar [13] H. Gao, J. F. Cai, Z. Shen and H. Zhao, Robust principal component analysis-based four-dimensional computed tomography,, Physics in Medicine and Biology, 56 (2011).  doi: 10.1088/0031-9155/56/11/002.  Google Scholar [14] R. Glowinski and P. Le Tallec, "Augmented Lagrangian and Operator Splitting Methods in Nonlinear Mechanics,", SIAM Studies in Applied Mathematics, 9 (1989).  doi: 10.1137/1.9781611970838.  Google Scholar [15] T. Goldstein and S. Osher, The split Bregman method for L1-regularized problems,, SIAM J. Imaging Sci., 2 (2009), 323.  doi: 10.1137/080725891.  Google Scholar [16] H. Jiang, Chengbo Li, Raziel Haimi-Cohen, Paul Wilford and Yin Zhang, Scalable video coding using compressive sensing,, Bell Labs Technical Journal, 16 (2012).   Google Scholar [17] H. Jiang, B. Mathews and P. Wilford, Compressive sensing for sound localization in wireless sensor network,, accepted for presentation at SENSORNET2012, (2012), 24.   Google Scholar [18] H. Jiang, Z. Shen, W. Deng and P. Wilford, Adaptive low rank and sparse decomposition in compressive sensing of surveillance video,, submitted, (2011).   Google Scholar [19] C. Li, H. Jiang, P. A., Wilford and Y. Zhang, Video coding using compressive sensing for wireless communications,, IEEE Wireless Communications and Networking Conference (WCNC), (2011), 2077.  doi: 10.1109/WCNC.2011.5779474.  Google Scholar [20] V. Mahadevan and N. Vasconcelos, Spatiotemporal saliency in highly dynamic scenes,, IEEE Trans. on Pattern Analysis and Machine Intelligence, 32 (2010), 171.  doi: 10.1109/TPAMI.2009.112.  Google Scholar [21] M. Piccardi, Background subtraction techniques: A review,, IEEE International Conference on Systems, 4 (2004), 3099.   Google Scholar [22] A. Ron and Z. Shen, Affine systems in $L_2(R^d)$: The analysis of the analysis operator,, Journal of Functional Analysis, 148 (1997), 408.  doi: 10.1006/jfan.1996.3079.  Google Scholar [23] M. Rudelson and R. Vershynin, On sparse reconstruction from Fourier and Gaussian measurements,, Communications on Pure and Applied Mathematics, 61 (2008), 1025.  doi: 10.1002/cpa.20227.  Google Scholar [24] Z. Shen, Wavelet frames and image restorations,, Proceedings of the International Congress of Mathematicians, IV (2010), 2834.   Google Scholar [25] C. Stauffer and W. E. L Grimson, Adaptive background mixture models for real-time tracking,, Computer Vision and Pattern Recognition, 2 (1999), 252.   Google Scholar [26] E. Sutter, The Fast $m$-Transform: A fast computation of cross-correlations with binary $m$-sequences,, SIAM J. Comput., 20 (1991), 686.  doi: 10.1137/0220043.  Google Scholar [27] , EC Funded CAVIAR project/IST 2001 37540,, 2003. Available from: \url{http://homepages.inf.ed.ac.uk/rbf/CAVIAR/}., ().   Google Scholar

show all references

##### References:
 [1] Y. Benezeth, P. M. Jodoin, B. Emile, H. Laurent and C. Rosenberger, Comparative study of background subtraction algorithms,, J. Electron. Imaging, 19 (2010).  doi: 10.1117/1.3456695.  Google Scholar [2] J.-F. Cai, E. J. Candès and Z. Shen, A singular value thresholding algorithm for matrix completion,, SIAM Journal on Optimization, 20 (2010), 1956.  doi: 10.1137/080738970.  Google Scholar [3] J.-F. Cai, S. Osher and Z. Shen, Split Bregman methods and frame based image restoration,, Multiscale Modeling and Simulation: A SIAM Interdisciplinary Journal, 8 (): 337.   Google Scholar [4] E.-J. Candès, X. Li, Y. Ma and J. Wright, Robust principal component analysis?,, Journal of ACM, 58 (2011).   Google Scholar [5] E. J. Candès, J. Romberg and T. Tao, Stable signal recovery from incomplete and inaccurate measurements,, Comm. Pure Appl. Math., 59 (2006), 1207.  doi: 10.1002/cpa.20124.  Google Scholar [6] V. Cevher, A. Sankaranarayanan, M. Duarte, D. Reddy, R. Baraniuk and R. Chellappa, Compressive sensing for background subtraction,, Computer Vision-ECCV 2008, (2008), 155.   Google Scholar [7] I. Daubechies, B. Han, A. Ron and Z. Shen, Framelets: MRA-based constructions of wavelet frames,, Applied and Computational Harmonic Analysis, 14 (2003), 1.  doi: 10.1016/S1063-5203(02)00511-0.  Google Scholar [8] W. Deng, W. Yin and Y. Zhang, "Group Sparse Optimization by Alternating Direction Method,", TR11-06, (2011), 11.   Google Scholar [9] B. Dong and Z. Shen, "MRA-Based Wavelet Frames and Applications,", IAS Lecture Notes Series, (2010).   Google Scholar [10] Y. Dong, G. N. DeSouza and T. X. Han, Illumination invariant foreground detection using multi-subspace learning,, International Journal of Knowledge-based and Intelligent Engineering Systems, 14 (2010), 31.   Google Scholar [11] D. Donoho, Compressed sensing,, IEEE Trans. on Information Theory, 52 (2006), 1289.  doi: 10.1109/TIT.2006.871582.  Google Scholar [12] M. Fornasier and H. Rauhut, Recovery algorithms for vector-valued data with joint sparsity constraints,, SIAM J. Numer. Anal., 46 (2008), 577.  doi: 10.1137/0606668909.  Google Scholar [13] H. Gao, J. F. Cai, Z. Shen and H. Zhao, Robust principal component analysis-based four-dimensional computed tomography,, Physics in Medicine and Biology, 56 (2011).  doi: 10.1088/0031-9155/56/11/002.  Google Scholar [14] R. Glowinski and P. Le Tallec, "Augmented Lagrangian and Operator Splitting Methods in Nonlinear Mechanics,", SIAM Studies in Applied Mathematics, 9 (1989).  doi: 10.1137/1.9781611970838.  Google Scholar [15] T. Goldstein and S. Osher, The split Bregman method for L1-regularized problems,, SIAM J. Imaging Sci., 2 (2009), 323.  doi: 10.1137/080725891.  Google Scholar [16] H. Jiang, Chengbo Li, Raziel Haimi-Cohen, Paul Wilford and Yin Zhang, Scalable video coding using compressive sensing,, Bell Labs Technical Journal, 16 (2012).   Google Scholar [17] H. Jiang, B. Mathews and P. Wilford, Compressive sensing for sound localization in wireless sensor network,, accepted for presentation at SENSORNET2012, (2012), 24.   Google Scholar [18] H. Jiang, Z. Shen, W. Deng and P. Wilford, Adaptive low rank and sparse decomposition in compressive sensing of surveillance video,, submitted, (2011).   Google Scholar [19] C. Li, H. Jiang, P. A., Wilford and Y. Zhang, Video coding using compressive sensing for wireless communications,, IEEE Wireless Communications and Networking Conference (WCNC), (2011), 2077.  doi: 10.1109/WCNC.2011.5779474.  Google Scholar [20] V. Mahadevan and N. Vasconcelos, Spatiotemporal saliency in highly dynamic scenes,, IEEE Trans. on Pattern Analysis and Machine Intelligence, 32 (2010), 171.  doi: 10.1109/TPAMI.2009.112.  Google Scholar [21] M. Piccardi, Background subtraction techniques: A review,, IEEE International Conference on Systems, 4 (2004), 3099.   Google Scholar [22] A. Ron and Z. Shen, Affine systems in $L_2(R^d)$: The analysis of the analysis operator,, Journal of Functional Analysis, 148 (1997), 408.  doi: 10.1006/jfan.1996.3079.  Google Scholar [23] M. Rudelson and R. Vershynin, On sparse reconstruction from Fourier and Gaussian measurements,, Communications on Pure and Applied Mathematics, 61 (2008), 1025.  doi: 10.1002/cpa.20227.  Google Scholar [24] Z. Shen, Wavelet frames and image restorations,, Proceedings of the International Congress of Mathematicians, IV (2010), 2834.   Google Scholar [25] C. Stauffer and W. E. L Grimson, Adaptive background mixture models for real-time tracking,, Computer Vision and Pattern Recognition, 2 (1999), 252.   Google Scholar [26] E. Sutter, The Fast $m$-Transform: A fast computation of cross-correlations with binary $m$-sequences,, SIAM J. Comput., 20 (1991), 686.  doi: 10.1137/0220043.  Google Scholar [27] , EC Funded CAVIAR project/IST 2001 37540,, 2003. Available from: \url{http://homepages.inf.ed.ac.uk/rbf/CAVIAR/}., ().   Google Scholar
 [1] Dan Zhu, Rosemary A. Renaut, Hongwei Li, Tianyou Liu. Fast non-convex low-rank matrix decomposition for separation of potential field data using minimal memory. Inverse Problems & Imaging, 2021, 15 (1) : 159-183. doi: 10.3934/ipi.2020076 [2] Mingchao Zhao, You-Wei Wen, Michael Ng, Hongwei Li. A nonlocal low rank model for poisson noise removal. Inverse Problems & Imaging, , () : -. doi: 10.3934/ipi.2021003 [3] Hui Gao, Jian Lv, Xiaoliang Wang, Liping Pang. An alternating linearization bundle method for a class of nonconvex optimization problem with inexact information. Journal of Industrial & Management Optimization, 2021, 17 (2) : 805-825. doi: 10.3934/jimo.2019135 [4] Peter Frolkovič, Viera Kleinová. A new numerical method for level set motion in normal direction used in optical flow estimation. Discrete & Continuous Dynamical Systems - S, 2021, 14 (3) : 851-863. doi: 10.3934/dcdss.2020347 [5] Xuemei Chen, Julia Dobrosotskaya. Inpainting via sparse recovery with directional constraints. Mathematical Foundations of Computing, 2020, 3 (4) : 229-247. doi: 10.3934/mfc.2020025 [6] Yujuan Li, Huaifu Wang, Peipei Zhou, Guoshuang Zhang. Some properties of the cycle decomposition of WG-NLFSR. Advances in Mathematics of Communications, 2021, 15 (1) : 155-165. doi: 10.3934/amc.2020050 [7] Darko Dimitrov, Hosam Abdo. Tight independent set neighborhood union condition for fractional critical deleted graphs and ID deleted graphs. Discrete & Continuous Dynamical Systems - S, 2019, 12 (4&5) : 711-721. doi: 10.3934/dcdss.2019045 [8] Abdelghafour Atlas, Mostafa Bendahmane, Fahd Karami, Driss Meskine, Omar Oubbih. A nonlinear fractional reaction-diffusion system applied to image denoising and decomposition. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020321 [9] Editorial Office. Retraction: Xiaohong Zhu, Lihe Zhou, Zili Yang and Joyati Debnath, A new text information extraction algorithm of video image under multimedia environment. Discrete & Continuous Dynamical Systems - S, 2019, 12 (4&5) : 1265-1265. doi: 10.3934/dcdss.2019087 [10] Meng Ding, Ting-Zhu Huang, Xi-Le Zhao, Michael K. Ng, Tian-Hui Ma. Tensor train rank minimization with nonlocal self-similarity for tensor completion. Inverse Problems & Imaging, , () : -. doi: 10.3934/ipi.2021001 [11] Ningyu Sha, Lei Shi, Ming Yan. Fast algorithms for robust principal component analysis with an upper bound on the rank. Inverse Problems & Imaging, 2021, 15 (1) : 109-128. doi: 10.3934/ipi.2020067 [12] Petr Čoupek, María J. Garrido-Atienza. Bilinear equations in Hilbert space driven by paths of low regularity. Discrete & Continuous Dynamical Systems - B, 2021, 26 (1) : 121-154. doi: 10.3934/dcdsb.2020230 [13] Sihem Guerarra. Maximum and minimum ranks and inertias of the Hermitian parts of the least rank solution of the matrix equation AXB = C. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 75-86. doi: 10.3934/naco.2020016 [14] Russell Ricks. The unique measure of maximal entropy for a compact rank one locally CAT(0) space. Discrete & Continuous Dynamical Systems - A, 2021, 41 (2) : 507-523. doi: 10.3934/dcds.2020266 [15] Kihoon Seong. Low regularity a priori estimates for the fourth order cubic nonlinear Schrödinger equation. Communications on Pure & Applied Analysis, 2020, 19 (12) : 5437-5473. doi: 10.3934/cpaa.2020247 [16] Maoding Zhen, Binlin Zhang, Vicenţiu D. Rădulescu. Normalized solutions for nonlinear coupled fractional systems: Low and high perturbations in the attractive case. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020379 [17] Youming Guo, Tingting Li. Optimal control strategies for an online game addiction model with low and high risk exposure. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020347 [18] Yi-Long Luo, Yangjun Ma. Low Mach number limit for the compressible inertial Qian-Sheng model of liquid crystals: Convergence for classical solutions. Discrete & Continuous Dynamical Systems - A, 2021, 41 (2) : 921-966. doi: 10.3934/dcds.2020304 [19] Zi Xu, Siwen Wang, Jinjin Huang. An efficient low complexity algorithm for box-constrained weighted maximin dispersion problem. Journal of Industrial & Management Optimization, 2021, 17 (2) : 971-979. doi: 10.3934/jimo.2020007 [20] Mehdi Bastani, Davod Khojasteh Salkuyeh. On the GSOR iteration method for image restoration. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 27-43. doi: 10.3934/naco.2020013

2019 Impact Factor: 1.373