
Previous Article
Variational source conditions and stability estimates for inverse electromagnetic medium scattering problems
 IPI Home
 This Issue

Next Article
Nonlinear Tikhonov regularization in Banach spaces for inverse scattering from anisotropic penetrable media
Foveated compressive imaging for low power vehicle fingerprinting and tracking in aerial imagery
HRL Laboratories LLC, 3011 Malibu Canyon Road, Malibu, CA 902654797, USA 
We describe a foveated compressive sensing approach for image analysis applications that utilizes knowledge of the task to be performed to reduce the number of required sensor measurements and sensor size, weight, and power (SWAP) compared to conventional Nyquist sampling and compressive sensingbased approaches. Our Compressive Optical Foveated Architecture (COFA) adapts the dictionary and compressive measurements to structure and sparsity in the signal, task, and scene by reducing measurement and dictionary mutual coherence and increasing sparsity using principles of actionable information and foveated compressive sensing. Actionable information is used to extract taskrelevant regions of interest (ROIs) from a lowresolution scene analysis by eliminating the effects of nuisances for occlusion and anomalous motion detection. From the extracted ROIs, preferential measurements are taken using foveation as part of the compressive sensing adaptation process. The taskspecific measurement matrix is optimized by using a novel saliencyweighted coherence minimization with respect to the learned signal dictionary. This incorporates the relative usage of the atoms in the dictionary. We utilize a patchbased method to learn the signal priors. A treestructured dictionary of image patches using KSVD is learned which can sparsely represent any given image patch with the tree structure. We have implemented COFA in an endtoend simulation of a vehicle fingerprinting task for aerial surveillance using foveated compressive measurements adapted to hierarchical ROIs consisting of background, roads, and vehicles. Our results show 113× reduction in measurements over conventional sensing and 28× reduction over compressive sensing using random measurements.
References:
[1] 
M. Aharon, M. Elad and A. Bruckstein, KSVD: An algorithm for designing overcomplete dictionaries for sparse representation, IEEE Transactions on Signal Processing, 54 (2006), 43114322. 
[2] 
A. Ayvaci, M. Raptis and S. Soatto, Occlusion Detection and Motion Estimation with Convex Optimization Neural Information Processing Systems, 2010. 
[3] 
A. Bruckstein, D. Donoho and M. Elad, From sparse solutions of systems of equations to sparse modeling of signals and images, SIAM Review, 51 (2009), 3481. doi: 10.1137/060657704. 
[4] 
E. Candés and T. Tao, Decoding by linear programming, IEEE Trans. Inform. Theory, 51 (2005), 42034215. doi: 10.1109/TIT.2005.858979. 
[5] 
I. Ciocoiu, Foveated compressed sensing, Proc. of Europe. Conf. on Circuit Theory and Design, (2011), 2932. doi: 10.1109/ECCTD.2011.6043336. 
[6] 
Columbus surrogate unmanned aerial vehicle (CSUAV) dataset, United States Air Force Research Lab (AFRL). 
[7] 
J. P. Curzan, C. R. Baxter and M. A. Massie, Variable acuity imager with dynamically steerable, programmable superpixels, Infrared Technology and Applications, Proc. SPIE, 4820 (2003), p318. doi: 10.1117/12.451183. 
[8] 
D. Donoho, A. Maleki and A. Montanari, Noise sensitivity phase transition in compressed sensing, IEEE Transactions on Information Theory, 57 (2011), 69206941. doi: 10.1109/TIT.2011.2165823. 
[9] 
J. DuarteCarvajalino and G. Sapiro, Learning to sense sparse signals: Simultaneous sensing matrix and sparsifying dictionary optimization, IEEE Transactions on Image Processing, 18 (2009), 13951408. doi: 10.1109/TIP.2009.2022459. 
[10] 
G. Georgiadis, A. Ayvaci and S. Soatto, Actionable Saliency Detection Proc. of CVPR, 2012. 
[11] 
Z. Harmany, A. Oh, R. Marcia and R. Willet, Motionadaptive compressive coded apertures, Proc. of SPIE, 8165 (2011), 15. doi: 10.1117/12.892726. 
[12] 
D. Heeger and A. Jepson, Subspace methods for recovering rigid motion, Intl. J. of Comp. Vis., 7 (1992), 95117. 
[13] 
InView Shortwave Infrared (SWIR) Cameras, http://inviewcorp.com/products/shortwaveinfraredswircameras/. 
[14] 
R. Jenatton, J. Mairal, G. Obozinski and F. Bach, Proximal Methods for Sparse Hierarchical Dictionary Learning J. Machine Learning Research, 2011. 
[15] 
R. Larcom and T. Coffman, Foveated image formation through compressive sensing, Proc. of Southwest Symp. Image Anal. Interp., (2010), 145148. doi: 10.1109/SSIAI.2010.5483896. 
[16] 
T. Mundhenk, K. Ni, K. Kim and Y. Owechko, Detection of unknown targets from aerial camera and extraction of simple object fingerprints for the purpose of target reacquisition, Proc. of SPIE, 8301 (2012), 114. doi: 10.1117/12.906491. 
[17] 
S. Soatto, Steps Towards a Theory of Visual Information Textbook Draft. 
[18] 
A. Soni and J. Haupt, Efficient adaptive compressive sensing using sparse hierarchical learned dictionaries, Proc. of ASILOMAR, (2011), 12501254. doi: 10.1109/ACSSC.2011.6190216. 
[19] 
P. D. Sturkie, Sturkie's Avian Physiology 5th Edition, Academic Press, San Diego. 
[20] 
N. Sundaram, T. Brox and K. Keutzer, Dense point trajectories by GPUaccelerated large displacement optical flow, Chapter: Computer Vision C ECCV 2010, Volume 6311 of the series Lecture Notes in Computer Science, (2010), 438451. doi: 10.1007/9783642155499_32. 
[21] 
F. Tanner, B. Colder, C. Pullen, D. Heagy, M. Eppolito, V. Carlan, C. Oertel and P. Sallee, Overhead Imagery Research Data Set: An Annotated Data Library and Tools to aid in the Development of Computer Vision Algorithms Proc. of IEEE Applied Imagery Pattern Rec. Workshop, 2009. doi: 10.1109/AIPR.2009.5466304. 
[22] 
L. ZelnikManor, K. Rosenblum and Y. Eldar, Sensing matrix optimization for blocksparse decoding, IEEE Transactions on Signal Processing, 59 (2011), 43004312. doi: 10.1109/TSP.2011.2159211. 
show all references
References:
[1] 
M. Aharon, M. Elad and A. Bruckstein, KSVD: An algorithm for designing overcomplete dictionaries for sparse representation, IEEE Transactions on Signal Processing, 54 (2006), 43114322. 
[2] 
A. Ayvaci, M. Raptis and S. Soatto, Occlusion Detection and Motion Estimation with Convex Optimization Neural Information Processing Systems, 2010. 
[3] 
A. Bruckstein, D. Donoho and M. Elad, From sparse solutions of systems of equations to sparse modeling of signals and images, SIAM Review, 51 (2009), 3481. doi: 10.1137/060657704. 
[4] 
E. Candés and T. Tao, Decoding by linear programming, IEEE Trans. Inform. Theory, 51 (2005), 42034215. doi: 10.1109/TIT.2005.858979. 
[5] 
I. Ciocoiu, Foveated compressed sensing, Proc. of Europe. Conf. on Circuit Theory and Design, (2011), 2932. doi: 10.1109/ECCTD.2011.6043336. 
[6] 
Columbus surrogate unmanned aerial vehicle (CSUAV) dataset, United States Air Force Research Lab (AFRL). 
[7] 
J. P. Curzan, C. R. Baxter and M. A. Massie, Variable acuity imager with dynamically steerable, programmable superpixels, Infrared Technology and Applications, Proc. SPIE, 4820 (2003), p318. doi: 10.1117/12.451183. 
[8] 
D. Donoho, A. Maleki and A. Montanari, Noise sensitivity phase transition in compressed sensing, IEEE Transactions on Information Theory, 57 (2011), 69206941. doi: 10.1109/TIT.2011.2165823. 
[9] 
J. DuarteCarvajalino and G. Sapiro, Learning to sense sparse signals: Simultaneous sensing matrix and sparsifying dictionary optimization, IEEE Transactions on Image Processing, 18 (2009), 13951408. doi: 10.1109/TIP.2009.2022459. 
[10] 
G. Georgiadis, A. Ayvaci and S. Soatto, Actionable Saliency Detection Proc. of CVPR, 2012. 
[11] 
Z. Harmany, A. Oh, R. Marcia and R. Willet, Motionadaptive compressive coded apertures, Proc. of SPIE, 8165 (2011), 15. doi: 10.1117/12.892726. 
[12] 
D. Heeger and A. Jepson, Subspace methods for recovering rigid motion, Intl. J. of Comp. Vis., 7 (1992), 95117. 
[13] 
InView Shortwave Infrared (SWIR) Cameras, http://inviewcorp.com/products/shortwaveinfraredswircameras/. 
[14] 
R. Jenatton, J. Mairal, G. Obozinski and F. Bach, Proximal Methods for Sparse Hierarchical Dictionary Learning J. Machine Learning Research, 2011. 
[15] 
R. Larcom and T. Coffman, Foveated image formation through compressive sensing, Proc. of Southwest Symp. Image Anal. Interp., (2010), 145148. doi: 10.1109/SSIAI.2010.5483896. 
[16] 
T. Mundhenk, K. Ni, K. Kim and Y. Owechko, Detection of unknown targets from aerial camera and extraction of simple object fingerprints for the purpose of target reacquisition, Proc. of SPIE, 8301 (2012), 114. doi: 10.1117/12.906491. 
[17] 
S. Soatto, Steps Towards a Theory of Visual Information Textbook Draft. 
[18] 
A. Soni and J. Haupt, Efficient adaptive compressive sensing using sparse hierarchical learned dictionaries, Proc. of ASILOMAR, (2011), 12501254. doi: 10.1109/ACSSC.2011.6190216. 
[19] 
P. D. Sturkie, Sturkie's Avian Physiology 5th Edition, Academic Press, San Diego. 
[20] 
N. Sundaram, T. Brox and K. Keutzer, Dense point trajectories by GPUaccelerated large displacement optical flow, Chapter: Computer Vision C ECCV 2010, Volume 6311 of the series Lecture Notes in Computer Science, (2010), 438451. doi: 10.1007/9783642155499_32. 
[21] 
F. Tanner, B. Colder, C. Pullen, D. Heagy, M. Eppolito, V. Carlan, C. Oertel and P. Sallee, Overhead Imagery Research Data Set: An Annotated Data Library and Tools to aid in the Development of Computer Vision Algorithms Proc. of IEEE Applied Imagery Pattern Rec. Workshop, 2009. doi: 10.1109/AIPR.2009.5466304. 
[22] 
L. ZelnikManor, K. Rosenblum and Y. Eldar, Sensing matrix optimization for blocksparse decoding, IEEE Transactions on Signal Processing, 59 (2011), 43004312. doi: 10.1109/TSP.2011.2159211. 
Method  Measurement  Dictionary 
rand + flat  random Gaussian orthonormal measurements  (flat) ksvd dictionary 
rand + tree  random Gaussian orthonormal measurements  hierarchical (tree) dictionary 
mc + flat  minimum coherence measurements  (flat) ksvd dictionary 
mc + tree  minimum coherence measurements  hierarchical (tree) dictionary 
wmc + tree  weighted minimum coherence measurements  hierarchical (tree) dictionary 
Method  Measurement  Dictionary 
rand + flat  random Gaussian orthonormal measurements  (flat) ksvd dictionary 
rand + tree  random Gaussian orthonormal measurements  hierarchical (tree) dictionary 
mc + flat  minimum coherence measurements  (flat) ksvd dictionary 
mc + tree  minimum coherence measurements  hierarchical (tree) dictionary 
wmc + tree  weighted minimum coherence measurements  hierarchical (tree) dictionary 
[1] 
Vikram Krishnamurthy, William Hoiles. Information diffusion in social sensing. Numerical Algebra, Control and Optimization, 2016, 6 (3) : 365411. doi: 10.3934/naco.2016017 
[2] 
Yangyang Xu, Wotao Yin, Stanley Osher. Learning circulant sensing kernels. Inverse Problems and Imaging, 2014, 8 (3) : 901923. doi: 10.3934/ipi.2014.8.901 
[3] 
JianWu Xue, XiaoKun Xu, Feng Zhang. Big data dynamic compressive sensing system architecture and optimization algorithm for internet of things. Discrete and Continuous Dynamical Systems  S, 2015, 8 (6) : 14011414. doi: 10.3934/dcdss.2015.8.1401 
[4] 
Hong Jiang, Wei Deng, Zuowei Shen. Surveillance video processing using compressive sensing. Inverse Problems and Imaging, 2012, 6 (2) : 201214. doi: 10.3934/ipi.2012.6.201 
[5] 
Zhihua Zhang, Naoki Saito. PHLST with adaptive tiling and its application to antarctic remote sensing image approximation. Inverse Problems and Imaging, 2014, 8 (1) : 321337. doi: 10.3934/ipi.2014.8.321 
[6] 
Yonggui Zhu, Yuying Shi, Bin Zhang, Xinyan Yu. Weightedaverage alternating minimization method for magnetic resonance image reconstruction based on compressive sensing. Inverse Problems and Imaging, 2014, 8 (3) : 925937. doi: 10.3934/ipi.2014.8.925 
[7] 
Meixin Xiong, Liuhong Chen, Ju Ming, Jaemin Shin. Accelerating the Bayesian inference of inverse problems by using datadriven compressive sensing method based on proper orthogonal decomposition. Electronic Research Archive, 2021, 29 (5) : 33833403. doi: 10.3934/era.2021044 
[8] 
Yingying Li, Stanley Osher. Coordinate descent optimization for l^{1} minimization with application to compressed sensing; a greedy algorithm. Inverse Problems and Imaging, 2009, 3 (3) : 487503. doi: 10.3934/ipi.2009.3.487 
[9] 
Jae Deok Kim, Ganguk Hwang. Crosslayer modeling and optimization of multichannel cognitive radio networks under imperfect channel sensing. Journal of Industrial and Management Optimization, 2015, 11 (3) : 807828. doi: 10.3934/jimo.2015.11.807 
[10] 
Qian Zhao, Bitao Jiang, Xiaogang Yu, Yue Zhang. Collaborative mission optimization for ship rapid search by multiple heterogeneous remote sensing satellites. Journal of Industrial and Management Optimization, 2021 doi: 10.3934/jimo.2021092 
[11] 
Steven L. Brunton, Joshua L. Proctor, Jonathan H. Tu, J. Nathan Kutz. Compressed sensing and dynamic mode decomposition. Journal of Computational Dynamics, 2015, 2 (2) : 165191. doi: 10.3934/jcd.2015002 
[12] 
Ying Zhang, Ling Ma, ZhengHai Huang. On phaseless compressed sensing with partially known support. Journal of Industrial and Management Optimization, 2020, 16 (3) : 15191526. doi: 10.3934/jimo.2019014 
[13] 
ZhiAn Wang. A kinetic chemotaxis model with internal states and temporal sensing. Kinetic and Related Models, 2022, 15 (1) : 2748. doi: 10.3934/krm.2021043 
[14] 
Richard Archibald, Hoang Tran. A dictionary learning algorithm for compression and reconstruction of streaming data in preset order. Discrete and Continuous Dynamical Systems  S, 2022, 15 (4) : 655668. doi: 10.3934/dcdss.2021102 
[15] 
Cesare Bracco, Annalisa Buffa, Carlotta Giannelli, Rafael Vázquez. Adaptive isogeometric methods with hierarchical splines: An overview. Discrete and Continuous Dynamical Systems, 2019, 39 (1) : 241261. doi: 10.3934/dcds.2019010 
[16] 
Prashant Shekhar, Abani Patra. Hierarchical approximations for data reduction and learning at multiple scales. Foundations of Data Science, 2020, 2 (2) : 123154. doi: 10.3934/fods.2020008 
[17] 
Shunfu Jin, Wuyi Yue, Shiying Ge. Equilibrium analysis of an opportunistic spectrum access mechanism with imperfect sensing results. Journal of Industrial and Management Optimization, 2017, 13 (3) : 12551271. doi: 10.3934/jimo.2016071 
[18] 
MinFan He, LiNing Xing, Wen Li, Shang Xiang, Xu Tan. Double layer programming model to the scheduling of remote sensing data processing tasks. Discrete and Continuous Dynamical Systems  S, 2019, 12 (4&5) : 15151526. doi: 10.3934/dcdss.2019104 
[19] 
Haruki Katayama, Hiroyuki Masuyama, Shoji Kasahara, Yutaka Takahashi. Effect of spectrum sensing overhead on performance for cognitive radio networks with channel bonding. Journal of Industrial and Management Optimization, 2014, 10 (1) : 2140. doi: 10.3934/jimo.2014.10.21 
[20] 
A Voutilainen, Jari P. Kaipio. Model reduction and pollution source identification from remote sensing data. Inverse Problems and Imaging, 2009, 3 (4) : 711730. doi: 10.3934/ipi.2009.3.711 
2020 Impact Factor: 1.639
Tools
Metrics
Other articles
by authors
[Back to Top]