# American Institute of Mathematical Sciences

doi: 10.3934/ipi.2021013

## Synthetic-Aperture Radar image based positioning in GPS-denied environments using Deep Cosine Similarity Neural Networks

 1 Department of Industrial and Systems Engineering, University of Florida, Gainesville, FL 32611, USA 2 Department of Information Systems and Analytics, Miami University, Oxford, OH 45056, USA 3 Air Force Research Laboratory (AFRL/RWWI), Eglin Air Force Base, FL 32542, USA

* Corresponding author: Maciej Rysz

Received  September 2020 Revised  November 2020 Published  January 2021

Fund Project: Distribution A: Approved for public release, distribution is unlimited, 96TW-2020-0170

Navigating unmanned aerial vehicles in precarious environments is of great importance. It is necessary to rely on alternative information processing techniques to attain spatial information that is required for navigation in such settings. This paper introduces a novel deep learning-based approach for navigating that exclusively relies on synthetic aperture radar (SAR) images. The proposed method utilizes deep neural networks (DNNs) for image matching, retrieval, and registration. To this end, we introduce Deep Cosine Similarity Neural Networks (DCSNNs) for mapping SAR images to a global descriptive feature vector. We also introduce a fine-tuning algorithm for DCSNNs, and DCSNNs are used to generate a database of feature vectors for SAR images that span a geographic area of interest, which, in turn, are compared against a feature vector of an inquiry image. Images similar to the inquiry are retrieved from the database by using a scalable distance measure between the feature vector outputs of DCSNN. Methods for reranking the retrieved SAR images that are used to update position coordinates of an inquiry SAR image by estimating from the best retrieved SAR image are also introduced. Numerical experiments comparing with baselines on the Polarimetric SAR (PolSAR) images are presented.

Citation: Seonho Park, Maciej Rysz, Kaitlin L. Fair, Panos M. Pardalos. Synthetic-Aperture Radar image based positioning in GPS-denied environments using Deep Cosine Similarity Neural Networks. Inverse Problems & Imaging, doi: 10.3934/ipi.2021013
##### References:

show all references

##### References:
Overview of the system for SAR aided navigating by SAR image representation, matching and registration
Example of a patch graph. (a) a SAR map from the UAVSAR dataset (b) patches extracted from the SAR map and its associated graph (c) visualization of the adjacency matrix. White cells show zero edge values whereas black cells show nonzero values
Overview of Product Quantization (PQ) with DCSNN
Comparison between keypoints generated by (a) SAR-SIFT and (b) SIFT
(a) SAR-SIFT based keypoints on two adjacent SAR patches (b) image matching of the keypoints via RANSAC
PolSAR data from UAVSAR dataset for our experiments. (From left) VVVV(R), HVHV(G), HHHH(B) channels, and total RGB image. Best viewed in color. (a) PolSAR map for Hayward Fault Zone in California, US. (b) PolSAR map for Yukon–Kuskokwim Delta in Alaska, US
Precision recall curves on Hayward Fault Zone (Top rows) and Yukon–Kuskokwim Delta (Bottom rows) PolSAR maps. From left column, the feature length is $24$, $48$, $96$, and $120$. AlexNet is used as a backbone
Examples of the retrieved SAR patches before and after reranking processes. First column represents examples of inquiry SAR patches. The first two rows ((a) and (b)) are from Hayward Fault Zone PolSAR and the later two rows ((c) and (d)) are from Yukon-Kuskokwim Delta PolSAR data. Having green box indicates it is correctly retrieved, whereas having red box indicates that it is incorrectly retrieved
Descriptions of the PolSAR map from UAVSAR [1]
 Region (Dataset Name) Usage Acquired Date Pixel Size (Height×Width) #Patches Hayward Fault Zone Training Oct 9, 2018 $23506\times3300$ 6440 Test May 30, 2019 $23476\times3300$ 6412 Yukon–Kuskokwim Delta Training Aug 28, 2018 $19066\times3300$ 5180 Test Sep 17, 2019 $19148\times3300$ 5208
 Region (Dataset Name) Usage Acquired Date Pixel Size (Height×Width) #Patches Hayward Fault Zone Training Oct 9, 2018 $23506\times3300$ 6440 Test May 30, 2019 $23476\times3300$ 6412 Yukon–Kuskokwim Delta Training Aug 28, 2018 $19066\times3300$ 5180 Test Sep 17, 2019 $19148\times3300$ 5208
Mean average precision (mAP) results of DCSNN and binary hashing methods before reranking on Hayward Fault Zone PolSAR map
 Methods Feature length $l$ AlexNet [21] VGG-11 [40] Pretrained (BH) $l=24$ 0.0478 0.0313 $l=48$ 0.0761 0.0522 $l=96$ 0.1547 0.1200 $l=120$ 0.1566 0.1219 Pretrained (PQ+AQD) $l=24$ 0.2332 0.1706 $l=48$ 0.3208 0.2055 $l=96$ 0.3231 0.2676 $l=120$ 0.3856 0.3161 DHN [50] $l=24$ 0.1182 0.1255 $l=48$ 0.1642 0.1693 $l=96$ 0.2274 0.2153 $l=120$ 0.3202 0.2449 DPSH [23] $l=24$ 0.0895 0.1220 $l=48$ 0.2825 0.2632 $l=96$ 0.4545 0.4005 $l=120$ 0.5213 0.4334 DHNN-L2 [24] $l=24$ 0.0451 0.1147 $l=48$ 0.0683 0.1291 $l=96$ 0.2044 0.1329 $l=120$ 0.2190 0.1304 DCSNN (ours) $l=24$ 0.2519 0.4889 $l=48$ 0.6145 0.6301 $l=96$ 0.6481 0.5783 $l=120$ 0.6813 0.5819
 Methods Feature length $l$ AlexNet [21] VGG-11 [40] Pretrained (BH) $l=24$ 0.0478 0.0313 $l=48$ 0.0761 0.0522 $l=96$ 0.1547 0.1200 $l=120$ 0.1566 0.1219 Pretrained (PQ+AQD) $l=24$ 0.2332 0.1706 $l=48$ 0.3208 0.2055 $l=96$ 0.3231 0.2676 $l=120$ 0.3856 0.3161 DHN [50] $l=24$ 0.1182 0.1255 $l=48$ 0.1642 0.1693 $l=96$ 0.2274 0.2153 $l=120$ 0.3202 0.2449 DPSH [23] $l=24$ 0.0895 0.1220 $l=48$ 0.2825 0.2632 $l=96$ 0.4545 0.4005 $l=120$ 0.5213 0.4334 DHNN-L2 [24] $l=24$ 0.0451 0.1147 $l=48$ 0.0683 0.1291 $l=96$ 0.2044 0.1329 $l=120$ 0.2190 0.1304 DCSNN (ours) $l=24$ 0.2519 0.4889 $l=48$ 0.6145 0.6301 $l=96$ 0.6481 0.5783 $l=120$ 0.6813 0.5819
mAP results of the DCSNN and binary hashing methods before reranking on Yukon–Kuskokwim Delta PolSAR map
 Methods Feature length $l$ AlexNet [21] VGG-11 [40] Pretrained (BH) $l=24$ 0.0566 0.0396 $l=48$ 0.1048 0.0550 $l=96$ 0.1927 0.1227 $l=120$ 0.2196 0.1174 Pretrained (PQ+AQD) $l=24$ 0.2684 0.1736 $l=48$ 0.3580 0.2091 $l=96$ 0.3718 0.2579 $l=120$ 0.4184 0.2690 DHN [50] $l=24$ 0.1219 0.1610 $l=48$ 0.2072 0.1910 $l=96$ 0.2715 0.2447 $l=120$ 0.3627 0.2239 DPSH [23] $l=24$ 0.1347 0.1170 $l=48$ 0.2371 0.2516 $l=96$ 0.3649 0.3281 $l=120$ 0.4098 0.3331 DHNN-L2 [24] $l=24$ 0.0621 0.1132 $l=48$ 0.1556 0.1812 $l=96$ 0.2916 0.3166 $l=120$ 0.3227 0.2822 DCSNN (ours) $l=24$ 0.4393 0.4324 $l=48$ 0.5424 0.5196 $l=96$ 0.5734 0.4913 $l=120$ 0.5996 0.4831
 Methods Feature length $l$ AlexNet [21] VGG-11 [40] Pretrained (BH) $l=24$ 0.0566 0.0396 $l=48$ 0.1048 0.0550 $l=96$ 0.1927 0.1227 $l=120$ 0.2196 0.1174 Pretrained (PQ+AQD) $l=24$ 0.2684 0.1736 $l=48$ 0.3580 0.2091 $l=96$ 0.3718 0.2579 $l=120$ 0.4184 0.2690 DHN [50] $l=24$ 0.1219 0.1610 $l=48$ 0.2072 0.1910 $l=96$ 0.2715 0.2447 $l=120$ 0.3627 0.2239 DPSH [23] $l=24$ 0.1347 0.1170 $l=48$ 0.2371 0.2516 $l=96$ 0.3649 0.3281 $l=120$ 0.4098 0.3331 DHNN-L2 [24] $l=24$ 0.0621 0.1132 $l=48$ 0.1556 0.1812 $l=96$ 0.2916 0.3166 $l=120$ 0.3227 0.2822 DCSNN (ours) $l=24$ 0.4393 0.4324 $l=48$ 0.5424 0.5196 $l=96$ 0.5734 0.4913 $l=120$ 0.5996 0.4831
mAP values before and after reranking with SAR-SIFT or SIFT on Hayward Fault Zone PolSAR map
 CNN backbone Feature length $l$ Before reranking After reranking (SAR-SIFT/SIFT) AlexNet [21] $l=24$ 0.2519 0.4074/0.3533 $l=48$ 0.6145 0.7394/0.6850 $l=96$ 0.6481 0.7548/0.6998 $l=120$ 0.6813 0.7760/0.7252 VGG-11 [40] $l=24$ 0.4889 0.6512/0.5813 $l=48$ 0.6301 0.7540/0.6923 $l=96$ 0.5783 0.6799/0.6231 $l=120$ 0.5819 0.6787/0.6216
 CNN backbone Feature length $l$ Before reranking After reranking (SAR-SIFT/SIFT) AlexNet [21] $l=24$ 0.2519 0.4074/0.3533 $l=48$ 0.6145 0.7394/0.6850 $l=96$ 0.6481 0.7548/0.6998 $l=120$ 0.6813 0.7760/0.7252 VGG-11 [40] $l=24$ 0.4889 0.6512/0.5813 $l=48$ 0.6301 0.7540/0.6923 $l=96$ 0.5783 0.6799/0.6231 $l=120$ 0.5819 0.6787/0.6216
mAP values of the DCSNN before and after reranking with SAR-SIFT or SIFT on Yukon–Kuskokwim Delta PolSAR map
 CNN backbone Feature length $l$ Before reranking After reranking (SAR-SIFT/SIFT) AlexNet [21] $l=24$ 0.4393 0.5831/0.5965 $l=48$ 0.5424 0.6591/0.6693 $l=96$ 0.5734 0.6782/0.6888 $l=120$ 0.5996 0.7021/0.7123 VGG-11 [40] $l=24$ 0.4324 0.5909/0.6030 $l=48$ 0.5196 0.6418/0.6521 $l=96$ 0.4913 0.5939/0.6036 $l=120$ 0.4831 0.5840/0.5940
 CNN backbone Feature length $l$ Before reranking After reranking (SAR-SIFT/SIFT) AlexNet [21] $l=24$ 0.4393 0.5831/0.5965 $l=48$ 0.5424 0.6591/0.6693 $l=96$ 0.5734 0.6782/0.6888 $l=120$ 0.5996 0.7021/0.7123 VGG-11 [40] $l=24$ 0.4324 0.5909/0.6030 $l=48$ 0.5196 0.6418/0.6521 $l=96$ 0.4913 0.5939/0.6036 $l=120$ 0.4831 0.5840/0.5940
Positioning accuracy examples
 Inquiry SAR Patch Actual Coordinates [$deg$] Estimated Coordinates [$deg$] Error [$m$] Fig. 8(a) 38.0625, -122.2733 38.0625, -122.2734 5.7288 Fig. 8(b) 37.9836, -122.3599 37.9836, -122.3600 5.7347 Fig. 8(c) 61.0926, -164.1878 61.0926, -164.1879 4.2529 Fig. 8(d) 61.0808, -164.1208 61.0808, -164.1208 5.0970
 Inquiry SAR Patch Actual Coordinates [$deg$] Estimated Coordinates [$deg$] Error [$m$] Fig. 8(a) 38.0625, -122.2733 38.0625, -122.2734 5.7288 Fig. 8(b) 37.9836, -122.3599 37.9836, -122.3600 5.7347 Fig. 8(c) 61.0926, -164.1878 61.0926, -164.1879 4.2529 Fig. 8(d) 61.0808, -164.1208 61.0808, -164.1208 5.0970
Mean and standard deviation of positioning error results
 Data Name Success Cases Ratio [%] Distance Error [$m$] Hayward Fault Zone 98.50 4.9635$\pm$0.1755 Yukon–Kuskokwim Delta 97.70 4.9522$\pm$0.4038
 Data Name Success Cases Ratio [%] Distance Error [$m$] Hayward Fault Zone 98.50 4.9635$\pm$0.1755 Yukon–Kuskokwim Delta 97.70 4.9522$\pm$0.4038
 [1] Zhihua Zhang, Naoki Saito. PHLST with adaptive tiling and its application to antarctic remote sensing image approximation. Inverse Problems & Imaging, 2014, 8 (1) : 321-337. doi: 10.3934/ipi.2014.8.321 [2] Jia Cai, Guanglong Xu, Zhensheng Hu. Sketch-based image retrieval via CAT loss with elastic net regularization. Mathematical Foundations of Computing, 2020, 3 (4) : 219-227. doi: 10.3934/mfc.2020013 [3] Mikhail Gilman, Semyon Tsynkov. Statistical characterization of scattering delay in synthetic aperture radar imaging. Inverse Problems & Imaging, 2020, 14 (3) : 511-533. doi: 10.3934/ipi.2020024 [4] Lekbir Afraites, Abdelghafour Atlas, Fahd Karami, Driss Meskine. Some class of parabolic systems applied to image processing. Discrete & Continuous Dynamical Systems - B, 2016, 21 (6) : 1671-1687. doi: 10.3934/dcdsb.2016017 [5] Juan Manuel Pastor, Javier García-Algarra, José M. Iriondo, José J. Ramasco, Javier Galeano. Dragging in mutualistic networks. Networks & Heterogeneous Media, 2015, 10 (1) : 37-52. doi: 10.3934/nhm.2015.10.37 [6] Braxton Osting, Jérôme Darbon, Stanley Osher. Statistical ranking using the $l^{1}$-norm on graphs. Inverse Problems & Imaging, 2013, 7 (3) : 907-926. doi: 10.3934/ipi.2013.7.907 [7] Cicely K. Macnamara, Mark A. J. Chaplain. Spatio-temporal models of synthetic genetic oscillators. Mathematical Biosciences & Engineering, 2017, 14 (1) : 249-262. doi: 10.3934/mbe.2017016 [8] Rui Hu, Yuan Yuan. Stability, bifurcation analysis in a neural network model with delay and diffusion. Conference Publications, 2009, 2009 (Special) : 367-376. doi: 10.3934/proc.2009.2009.367 [9] Y. Latushkin, B. Layton. The optimal gap condition for invariant manifolds. Discrete & Continuous Dynamical Systems - A, 1999, 5 (2) : 233-268. doi: 10.3934/dcds.1999.5.233 [10] Alessandro Gondolo, Fernando Guevara Vasquez. Characterization and synthesis of Rayleigh damped elastodynamic networks. Networks & Heterogeneous Media, 2014, 9 (2) : 299-314. doi: 10.3934/nhm.2014.9.299 [11] Juan Manuel Pastor, Javier García-Algarra, Javier Galeano, José María Iriondo, José J. Ramasco. A simple and bounded model of population dynamics for mutualistic networks. Networks & Heterogeneous Media, 2015, 10 (1) : 53-70. doi: 10.3934/nhm.2015.10.53 [12] Christopher Bose, Rua Murray. Minimum 'energy' approximations of invariant measures for nonsingular transformations. Discrete & Continuous Dynamical Systems - A, 2006, 14 (3) : 597-615. doi: 10.3934/dcds.2006.14.597 [13] Boris Kramer, John R. Singler. A POD projection method for large-scale algebraic Riccati equations. Numerical Algebra, Control & Optimization, 2016, 6 (4) : 413-435. doi: 10.3934/naco.2016018 [14] Hong Seng Sim, Wah June Leong, Chuei Yee Chen, Siti Nur Iqmal Ibrahim. Multi-step spectral gradient methods with modified weak secant relation for large scale unconstrained optimization. Numerical Algebra, Control & Optimization, 2018, 8 (3) : 377-387. doi: 10.3934/naco.2018024 [15] Longxiang Fang, Narayanaswamy Balakrishnan, Wenyu Huang. Stochastic comparisons of parallel systems with scale proportional hazards components equipped with starting devices. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2021004 [16] Marian Gidea, Rafael de la Llave, Tere M. Seara. A general mechanism of instability in Hamiltonian systems: Skipping along a normally hyperbolic invariant manifold. Discrete & Continuous Dynamical Systems - A, 2020, 40 (12) : 6795-6813. doi: 10.3934/dcds.2020166 [17] Manfred Einsiedler, Elon Lindenstrauss. On measures invariant under diagonalizable actions: the Rank-One case and the general Low-Entropy method. Journal of Modern Dynamics, 2008, 2 (1) : 83-128. doi: 10.3934/jmd.2008.2.83

2019 Impact Factor: 1.373

## Tools

Article outline

Figures and Tables