
-
Previous Article
Variational source conditions and stability estimates for inverse electromagnetic medium scattering problems
- IPI Home
- This Issue
-
Next Article
Non-linear Tikhonov regularization in Banach spaces for inverse scattering from anisotropic penetrable media
Foveated compressive imaging for low power vehicle fingerprinting and tracking in aerial imagery
HRL Laboratories LLC, 3011 Malibu Canyon Road, Malibu, CA 90265-4797, USA |
We describe a foveated compressive sensing approach for image analysis applications that utilizes knowledge of the task to be performed to reduce the number of required sensor measurements and sensor size, weight, and power (SWAP) compared to conventional Nyquist sampling and compressive sensing-based approaches. Our Compressive Optical Foveated Architecture (COFA) adapts the dictionary and compressive measurements to structure and sparsity in the signal, task, and scene by reducing measurement and dictionary mutual coherence and increasing sparsity using principles of actionable information and foveated compressive sensing. Actionable information is used to extract task-relevant regions of interest (ROIs) from a low-resolution scene analysis by eliminating the effects of nuisances for occlusion and anomalous motion detection. From the extracted ROIs, preferential measurements are taken using foveation as part of the compressive sensing adaptation process. The task-specific measurement matrix is optimized by using a novel saliency-weighted coherence minimization with respect to the learned signal dictionary. This incorporates the relative usage of the atoms in the dictionary. We utilize a patch-based method to learn the signal priors. A tree-structured dictionary of image patches using K-SVD is learned which can sparsely represent any given image patch with the tree structure. We have implemented COFA in an end-to-end simulation of a vehicle fingerprinting task for aerial surveillance using foveated compressive measurements adapted to hierarchical ROIs consisting of background, roads, and vehicles. Our results show 113× reduction in measurements over conventional sensing and 28× reduction over compressive sensing using random measurements.
References:
[1] |
M. Aharon, M. Elad and A. Bruckstein,
K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation, IEEE Transactions on Signal Processing, 54 (2006), 4311-4322.
|
[2] |
A. Ayvaci, M. Raptis and S. Soatto,
Occlusion Detection and Motion Estimation with Convex Optimization Neural Information Processing Systems, 2010. |
[3] |
A. Bruckstein, D. Donoho and M. Elad,
From sparse solutions of systems of equations to sparse modeling of signals and images, SIAM Review, 51 (2009), 34-81.
doi: 10.1137/060657704. |
[4] |
E. Candés and T. Tao,
Decoding by linear programming, IEEE Trans. Inform. Theory, 51 (2005), 4203-4215.
doi: 10.1109/TIT.2005.858979. |
[5] |
I. Ciocoiu,
Foveated compressed sensing, Proc. of Europe. Conf. on Circuit Theory and Design, (2011), 29-32.
doi: 10.1109/ECCTD.2011.6043336. |
[6] |
Columbus surrogate unmanned aerial vehicle (CSUAV) dataset, United States Air Force Research Lab (AFRL). |
[7] |
J. P. Curzan, C. R. Baxter and M. A. Massie,
Variable acuity imager with dynamically steerable, programmable superpixels, Infrared Technology and Applications, Proc. SPIE, 4820 (2003), p318.
doi: 10.1117/12.451183. |
[8] |
D. Donoho, A. Maleki and A. Montanari,
Noise sensitivity phase transition in compressed sensing, IEEE Transactions on Information Theory, 57 (2011), 6920-6941.
doi: 10.1109/TIT.2011.2165823. |
[9] |
J. Duarte-Carvajalino and G. Sapiro,
Learning to sense sparse signals: Simultaneous sensing matrix and sparsifying dictionary optimization, IEEE Transactions on Image Processing, 18 (2009), 1395-1408.
doi: 10.1109/TIP.2009.2022459. |
[10] |
G. Georgiadis, A. Ayvaci and S. Soatto,
Actionable Saliency Detection Proc. of CVPR, 2012. |
[11] |
Z. Harmany, A. Oh, R. Marcia and R. Willet,
Motion-adaptive compressive coded apertures, Proc. of SPIE, 8165 (2011), 1-5.
doi: 10.1117/12.892726. |
[12] |
D. Heeger and A. Jepson,
Subspace methods for recovering rigid motion, Intl. J. of Comp. Vis., 7 (1992), 95-117.
|
[13] |
InView Shortwave Infrared (SWIR) Cameras, http://inviewcorp.com/products/shortwave-infrared-swir-cameras/. |
[14] |
R. Jenatton, J. Mairal, G. Obozinski and F. Bach,
Proximal Methods for Sparse Hierarchical Dictionary Learning J. Machine Learning Research, 2011. |
[15] |
R. Larcom and T. Coffman,
Foveated image formation through compressive sensing, Proc. of Southwest Symp. Image Anal. Interp., (2010), 145-148.
doi: 10.1109/SSIAI.2010.5483896. |
[16] |
T. Mundhenk, K. Ni, K. Kim and Y. Owechko,
Detection of unknown targets from aerial camera and extraction of simple object fingerprints for the purpose of target reacquisition, Proc. of SPIE, 8301 (2012), 1-14.
doi: 10.1117/12.906491. |
[17] |
S. Soatto,
Steps Towards a Theory of Visual Information Textbook Draft. |
[18] |
A. Soni and J. Haupt,
Efficient adaptive compressive sensing using sparse hierarchical learned dictionaries, Proc. of ASILOMAR, (2011), 1250-1254.
doi: 10.1109/ACSSC.2011.6190216. |
[19] |
P. D. Sturkie,
Sturkie's Avian Physiology 5th Edition, Academic Press, San Diego. |
[20] |
N. Sundaram, T. Brox and K. Keutzer, Dense point trajectories by GPU-accelerated large
displacement optical flow, Chapter: Computer Vision C ECCV 2010, Volume 6311 of the
series Lecture Notes in Computer Science, (2010), 438-451.
doi: 10.1007/978-3-642-15549-9_32. |
[21] |
F. Tanner, B. Colder, C. Pullen, D. Heagy, M. Eppolito, V. Carlan, C. Oertel and P. Sallee,
Overhead Imagery Research Data Set: An Annotated Data Library and Tools to aid in the Development of Computer Vision Algorithms Proc. of IEEE Applied Imagery Pattern Rec. Workshop, 2009.
doi: 10.1109/AIPR.2009.5466304. |
[22] |
L. Zelnik-Manor, K. Rosenblum and Y. Eldar,
Sensing matrix optimization for block-sparse decoding, IEEE Transactions on Signal Processing, 59 (2011), 4300-4312.
doi: 10.1109/TSP.2011.2159211. |
show all references
References:
[1] |
M. Aharon, M. Elad and A. Bruckstein,
K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation, IEEE Transactions on Signal Processing, 54 (2006), 4311-4322.
|
[2] |
A. Ayvaci, M. Raptis and S. Soatto,
Occlusion Detection and Motion Estimation with Convex Optimization Neural Information Processing Systems, 2010. |
[3] |
A. Bruckstein, D. Donoho and M. Elad,
From sparse solutions of systems of equations to sparse modeling of signals and images, SIAM Review, 51 (2009), 34-81.
doi: 10.1137/060657704. |
[4] |
E. Candés and T. Tao,
Decoding by linear programming, IEEE Trans. Inform. Theory, 51 (2005), 4203-4215.
doi: 10.1109/TIT.2005.858979. |
[5] |
I. Ciocoiu,
Foveated compressed sensing, Proc. of Europe. Conf. on Circuit Theory and Design, (2011), 29-32.
doi: 10.1109/ECCTD.2011.6043336. |
[6] |
Columbus surrogate unmanned aerial vehicle (CSUAV) dataset, United States Air Force Research Lab (AFRL). |
[7] |
J. P. Curzan, C. R. Baxter and M. A. Massie,
Variable acuity imager with dynamically steerable, programmable superpixels, Infrared Technology and Applications, Proc. SPIE, 4820 (2003), p318.
doi: 10.1117/12.451183. |
[8] |
D. Donoho, A. Maleki and A. Montanari,
Noise sensitivity phase transition in compressed sensing, IEEE Transactions on Information Theory, 57 (2011), 6920-6941.
doi: 10.1109/TIT.2011.2165823. |
[9] |
J. Duarte-Carvajalino and G. Sapiro,
Learning to sense sparse signals: Simultaneous sensing matrix and sparsifying dictionary optimization, IEEE Transactions on Image Processing, 18 (2009), 1395-1408.
doi: 10.1109/TIP.2009.2022459. |
[10] |
G. Georgiadis, A. Ayvaci and S. Soatto,
Actionable Saliency Detection Proc. of CVPR, 2012. |
[11] |
Z. Harmany, A. Oh, R. Marcia and R. Willet,
Motion-adaptive compressive coded apertures, Proc. of SPIE, 8165 (2011), 1-5.
doi: 10.1117/12.892726. |
[12] |
D. Heeger and A. Jepson,
Subspace methods for recovering rigid motion, Intl. J. of Comp. Vis., 7 (1992), 95-117.
|
[13] |
InView Shortwave Infrared (SWIR) Cameras, http://inviewcorp.com/products/shortwave-infrared-swir-cameras/. |
[14] |
R. Jenatton, J. Mairal, G. Obozinski and F. Bach,
Proximal Methods for Sparse Hierarchical Dictionary Learning J. Machine Learning Research, 2011. |
[15] |
R. Larcom and T. Coffman,
Foveated image formation through compressive sensing, Proc. of Southwest Symp. Image Anal. Interp., (2010), 145-148.
doi: 10.1109/SSIAI.2010.5483896. |
[16] |
T. Mundhenk, K. Ni, K. Kim and Y. Owechko,
Detection of unknown targets from aerial camera and extraction of simple object fingerprints for the purpose of target reacquisition, Proc. of SPIE, 8301 (2012), 1-14.
doi: 10.1117/12.906491. |
[17] |
S. Soatto,
Steps Towards a Theory of Visual Information Textbook Draft. |
[18] |
A. Soni and J. Haupt,
Efficient adaptive compressive sensing using sparse hierarchical learned dictionaries, Proc. of ASILOMAR, (2011), 1250-1254.
doi: 10.1109/ACSSC.2011.6190216. |
[19] |
P. D. Sturkie,
Sturkie's Avian Physiology 5th Edition, Academic Press, San Diego. |
[20] |
N. Sundaram, T. Brox and K. Keutzer, Dense point trajectories by GPU-accelerated large
displacement optical flow, Chapter: Computer Vision C ECCV 2010, Volume 6311 of the
series Lecture Notes in Computer Science, (2010), 438-451.
doi: 10.1007/978-3-642-15549-9_32. |
[21] |
F. Tanner, B. Colder, C. Pullen, D. Heagy, M. Eppolito, V. Carlan, C. Oertel and P. Sallee,
Overhead Imagery Research Data Set: An Annotated Data Library and Tools to aid in the Development of Computer Vision Algorithms Proc. of IEEE Applied Imagery Pattern Rec. Workshop, 2009.
doi: 10.1109/AIPR.2009.5466304. |
[22] |
L. Zelnik-Manor, K. Rosenblum and Y. Eldar,
Sensing matrix optimization for block-sparse decoding, IEEE Transactions on Signal Processing, 59 (2011), 4300-4312.
doi: 10.1109/TSP.2011.2159211. |




















Method | Measurement | Dictionary |
rand + flat | random Gaussian orthonormal measurements | (flat) ksvd dictionary |
rand + tree | random Gaussian orthonormal measurements | hierarchical (tree) dictionary |
mc + flat | minimum coherence measurements | (flat) ksvd dictionary |
mc + tree | minimum coherence measurements | hierarchical (tree) dictionary |
wmc + tree | weighted minimum coherence measurements | hierarchical (tree) dictionary |
Method | Measurement | Dictionary |
rand + flat | random Gaussian orthonormal measurements | (flat) ksvd dictionary |
rand + tree | random Gaussian orthonormal measurements | hierarchical (tree) dictionary |
mc + flat | minimum coherence measurements | (flat) ksvd dictionary |
mc + tree | minimum coherence measurements | hierarchical (tree) dictionary |
wmc + tree | weighted minimum coherence measurements | hierarchical (tree) dictionary |
[1] |
Vikram Krishnamurthy, William Hoiles. Information diffusion in social sensing. Numerical Algebra, Control and Optimization, 2016, 6 (3) : 365-411. doi: 10.3934/naco.2016017 |
[2] |
Yangyang Xu, Wotao Yin, Stanley Osher. Learning circulant sensing kernels. Inverse Problems and Imaging, 2014, 8 (3) : 901-923. doi: 10.3934/ipi.2014.8.901 |
[3] |
Jian-Wu Xue, Xiao-Kun Xu, Feng Zhang. Big data dynamic compressive sensing system architecture and optimization algorithm for internet of things. Discrete and Continuous Dynamical Systems - S, 2015, 8 (6) : 1401-1414. doi: 10.3934/dcdss.2015.8.1401 |
[4] |
Hong Jiang, Wei Deng, Zuowei Shen. Surveillance video processing using compressive sensing. Inverse Problems and Imaging, 2012, 6 (2) : 201-214. doi: 10.3934/ipi.2012.6.201 |
[5] |
Zhihua Zhang, Naoki Saito. PHLST with adaptive tiling and its application to antarctic remote sensing image approximation. Inverse Problems and Imaging, 2014, 8 (1) : 321-337. doi: 10.3934/ipi.2014.8.321 |
[6] |
Yonggui Zhu, Yuying Shi, Bin Zhang, Xinyan Yu. Weighted-average alternating minimization method for magnetic resonance image reconstruction based on compressive sensing. Inverse Problems and Imaging, 2014, 8 (3) : 925-937. doi: 10.3934/ipi.2014.8.925 |
[7] |
Meixin Xiong, Liuhong Chen, Ju Ming, Jaemin Shin. Accelerating the Bayesian inference of inverse problems by using data-driven compressive sensing method based on proper orthogonal decomposition. Electronic Research Archive, 2021, 29 (5) : 3383-3403. doi: 10.3934/era.2021044 |
[8] |
Yingying Li, Stanley Osher. Coordinate descent optimization for l1 minimization with application to compressed sensing; a greedy algorithm. Inverse Problems and Imaging, 2009, 3 (3) : 487-503. doi: 10.3934/ipi.2009.3.487 |
[9] |
Jae Deok Kim, Ganguk Hwang. Cross-layer modeling and optimization of multi-channel cognitive radio networks under imperfect channel sensing. Journal of Industrial and Management Optimization, 2015, 11 (3) : 807-828. doi: 10.3934/jimo.2015.11.807 |
[10] |
Qian Zhao, Bitao Jiang, Xiaogang Yu, Yue Zhang. Collaborative mission optimization for ship rapid search by multiple heterogeneous remote sensing satellites. Journal of Industrial and Management Optimization, 2021 doi: 10.3934/jimo.2021092 |
[11] |
Steven L. Brunton, Joshua L. Proctor, Jonathan H. Tu, J. Nathan Kutz. Compressed sensing and dynamic mode decomposition. Journal of Computational Dynamics, 2015, 2 (2) : 165-191. doi: 10.3934/jcd.2015002 |
[12] |
Ying Zhang, Ling Ma, Zheng-Hai Huang. On phaseless compressed sensing with partially known support. Journal of Industrial and Management Optimization, 2020, 16 (3) : 1519-1526. doi: 10.3934/jimo.2019014 |
[13] |
Zhi-An Wang. A kinetic chemotaxis model with internal states and temporal sensing. Kinetic and Related Models, 2022, 15 (1) : 27-48. doi: 10.3934/krm.2021043 |
[14] |
Richard Archibald, Hoang Tran. A dictionary learning algorithm for compression and reconstruction of streaming data in preset order. Discrete and Continuous Dynamical Systems - S, 2022, 15 (4) : 655-668. doi: 10.3934/dcdss.2021102 |
[15] |
Cesare Bracco, Annalisa Buffa, Carlotta Giannelli, Rafael Vázquez. Adaptive isogeometric methods with hierarchical splines: An overview. Discrete and Continuous Dynamical Systems, 2019, 39 (1) : 241-261. doi: 10.3934/dcds.2019010 |
[16] |
Prashant Shekhar, Abani Patra. Hierarchical approximations for data reduction and learning at multiple scales. Foundations of Data Science, 2020, 2 (2) : 123-154. doi: 10.3934/fods.2020008 |
[17] |
Shunfu Jin, Wuyi Yue, Shiying Ge. Equilibrium analysis of an opportunistic spectrum access mechanism with imperfect sensing results. Journal of Industrial and Management Optimization, 2017, 13 (3) : 1255-1271. doi: 10.3934/jimo.2016071 |
[18] |
Min-Fan He, Li-Ning Xing, Wen Li, Shang Xiang, Xu Tan. Double layer programming model to the scheduling of remote sensing data processing tasks. Discrete and Continuous Dynamical Systems - S, 2019, 12 (4&5) : 1515-1526. doi: 10.3934/dcdss.2019104 |
[19] |
Haruki Katayama, Hiroyuki Masuyama, Shoji Kasahara, Yutaka Takahashi. Effect of spectrum sensing overhead on performance for cognitive radio networks with channel bonding. Journal of Industrial and Management Optimization, 2014, 10 (1) : 21-40. doi: 10.3934/jimo.2014.10.21 |
[20] |
A Voutilainen, Jari P. Kaipio. Model reduction and pollution source identification from remote sensing data. Inverse Problems and Imaging, 2009, 3 (4) : 711-730. doi: 10.3934/ipi.2009.3.711 |
2020 Impact Factor: 1.639
Tools
Metrics
Other articles
by authors
[Back to Top]