# American Institute of Mathematical Sciences

November  2019, 2(4): 299-314. doi: 10.3934/mfc.2019019

## Big Map R-CNN for object detection in large-scale remote sensing images

 a. FIST LAB, School of Information Science and Engineering, Yunnan University Kunming, 650091, Yunnan, China b. Yunnan Union Vision Technology Co Ltd. Kunming, 650091, Yunnan, China c. School of Software, Yunnan University Kunming, Yunnan University Kunming, 650091, Yunnan, China

* Corresponding author: Dapeng Tao

Published  December 2019

Detecting sparse and multi-sized objects in very high resolution (VHR) remote sensing images remains a significant challenge in satellite imagery applications and analytics. Difficulties include broad geographical scene distributions and high pixel counts in each image: a large-scale satellite image contains tens to hundreds of millions of pixels and dozens of complex backgrounds. Furthermore, the scale of the same category object can vary widely (e.g., ships can measure from several to thousands of pixels). To address these issues, here we propose the Big Map R-CNN method to improve object detection in VHR satellite imagery. Big Map R-CNN introduces mean shift clustering for quadric detecting based on the existing Mask R-CNN architecture. Big Map R-CNN considers four main aspects: 1) big map cropping to generate small size sub-images; 2) detecting these sub-images using the typical Mask R-CNN network; 3) screening out fragmented low-confidence targets and collecting uncertain image regions by clustering; 4) quadric detecting to generate prediction boxes. We also introduce a new large-scale and VHR remote sensing imagery dataset containing two categories (RSI LS-VHR-2) for detection performance verification. Comprehensive evaluations on RSI LS-VHR-2 dataset demonstrate the effectiveness of the proposed Big Map R-CNN algorithm for object detection in large-scale remote sensing images.

Citation: Linfei Wang, Dapeng Tao, Ruonan Wang, Ruxin Wang, Hao Li. Big Map R-CNN for object detection in large-scale remote sensing images. Mathematical Foundations of Computing, 2019, 2 (4) : 299-314. doi: 10.3934/mfc.2019019
##### References:

show all references

##### References:
Motivation for the proposed method. (a) Remote sensing scene of Madrid Airport. (b) Remote sensing scene of the South China Sea. These examples are from the RSI LS-VHR-2 dataset. The targets in the images are indicated by red cicles. The remote sensing scenes show the characteristics of large scale, high resolution, and relatively sparse target distribution, which means that existing methods are suboptimal for detection
The scheme of Big Map R-CNN, containing three main components: 1) cropping the input big map in the form of a sliding window; 2) detecting each sub-image sequentially and filtering possible object areas; 3) using mean shift clustering to precisely locate candidate object areas, cropping the new sub-images containing possible objects, and using quadric-detecting to judge whether there is an object or not
Large-scale image cropping
PRCs of the proposed Big Map R-CNN method and three other state-of-the-art detection methods (YOLOv3, Faster R-CNN, and Mask R-CNN). (a) is the PRC of the four methods for aircraft when IoU = 0.5; (b) is the PRC of the four methods for aircraft when IoU = 0.75; (c) is the PRC of the four methods for ships when IoU = 0.5; (d) is the PRC of the four methods for ships when IoU = 0.75
Some examples from the RSI LS-VHR-2 dataset
Detection comparisons of the different methods. (a) Typical Mask R-CNN for aircraft; (b) Big Map R-CNN for aircraft; (c) typical Mask R-CNN for ships; (d) Big Map R-CNN for ships. The true positives are indicated by green rectangles, the false negatives are indicated by red circles, and the bounding boxes that deviate from the ground truth are indicated by red rectangles
DESCRIPTION OF THE RSI LS-VHR-2 DATASET
 Label Name Total instances Complete instances Fragmentary instances Scene class Images Image width Sub-images 1 aircraft 103917 85975 17942 203 2858 6000-15000 62129 2 ship 68436 54386 14050 30 397 5000-18000 53860
 Label Name Total instances Complete instances Fragmentary instances Scene class Images Image width Sub-images 1 aircraft 103917 85975 17942 203 2858 6000-15000 62129 2 ship 68436 54386 14050 30 397 5000-18000 53860
DETAILS OF THE TEST IMAGES
 Label Scale(pixels) Images Instances Sub-images aircraft $8000\times8000$ 5 272 980 ship $8000\times8000$ 5 225 980
 Label Scale(pixels) Images Instances Sub-images aircraft $8000\times8000$ 5 272 980 ship $8000\times8000$ 5 225 980
PARAMETER SETTING OF Mask R-CNN AND Big Map R-CNN
 Input Size Per Batch Size Max Iteration Anchor Stride Base Learning Rate Steps Weight Decay NMS Threshold Momentum 600 8 90000 (4, 8, 16, 32, 64) 0.01 (60000, 80000) 0.0001 0.7 0.9
 Input Size Per Batch Size Max Iteration Anchor Stride Base Learning Rate Steps Weight Decay NMS Threshold Momentum 600 8 90000 (4, 8, 16, 32, 64) 0.01 (60000, 80000) 0.0001 0.7 0.9
PERFARMANCE COMPARISONS OF THREE DIFFERENT CROPPING SIZE IN Faster R-CNN NETWORK
 Cropping Size AP Cost time(s) C300 0.430 45.82 C600 0.651 13.20 C800 0.647 8.79
 Cropping Size AP Cost time(s) C300 0.430 45.82 C600 0.651 13.20 C800 0.647 8.79
PERFORMANCE COMPARISONS OF THE FOUR METHODS ON AIRCRAFT
 Method IoU=0.5 IoU=0.75 TP FP FN Recall Precision AP TP FP FN Recall Precision AP YOLOv3 213 25 59 0.783 0.895 0.727 166 72 106 0.610 0.6974 0.494 Faster R-CNN 242 55 30 0.890 0.815 0.830 189 108 83 0.695 0.636 0.618 Mask R-CNN 245 38 27 0.901 0.866 0.843 184 99 88 0.676 0.650 0.570 Big Map R-CNN 261 4 11 0.960 0.985 0.959 241 24 31 0.886 0.909 0.850
 Method IoU=0.5 IoU=0.75 TP FP FN Recall Precision AP TP FP FN Recall Precision AP YOLOv3 213 25 59 0.783 0.895 0.727 166 72 106 0.610 0.6974 0.494 Faster R-CNN 242 55 30 0.890 0.815 0.830 189 108 83 0.695 0.636 0.618 Mask R-CNN 245 38 27 0.901 0.866 0.843 184 99 88 0.676 0.650 0.570 Big Map R-CNN 261 4 11 0.960 0.985 0.959 241 24 31 0.886 0.909 0.850
PERFORMANCE COMPARISONS OF THE FOUR METHODS ON SHIP
 Method IoU=0.5 IoU=0.75 TP FP FN Recall Precision AP TP FP FN Recall Precision AP YOLOv3 128 53 97 0.569 0.707 0.513 66 115 159 0.293 0.365 0.213 Faster R-CNN 164 185 61 0.729 0.470 0.651 78 271 147 0.347 0.223 0.259 Mask R-CNN 166 121 59 0.738 0.578 0.661 78 209 147 0.347 0.272 0.273 Big Map R-CNN 191 49 34 0.849 0.796 0.826 133 107 92 0.591 0.554 0.546
 Method IoU=0.5 IoU=0.75 TP FP FN Recall Precision AP TP FP FN Recall Precision AP YOLOv3 128 53 97 0.569 0.707 0.513 66 115 159 0.293 0.365 0.213 Faster R-CNN 164 185 61 0.729 0.470 0.651 78 271 147 0.347 0.223 0.259 Mask R-CNN 166 121 59 0.738 0.578 0.661 78 209 147 0.347 0.272 0.273 Big Map R-CNN 191 49 34 0.849 0.796 0.826 133 107 92 0.591 0.554 0.546
THE AVERAGE PRECISION OF Mask R-CNN AND Big Map R-CNN IN RSI LS-VHR-2 DATASET
 Method Backbone AP($\%$) Mask R-CNN ResNet50 75.2 Big Map R-CNN ResNet50 89.2
 Method Backbone AP($\%$) Mask R-CNN ResNet50 75.2 Big Map R-CNN ResNet50 89.2
COMPREHENSIVE PERFORMANCE COMPARISONS OF FOUR METHODS
 Method mAP (IoU=0.5) mAP (IoU=0.75) Inference time(s/im) YOLOv3 0.620 0.354 3.310 Faster R-CNN 0.741 0.439 13.254 Mask R-CNN 0.752 0.422 13.310 Big Map R-CNN 0.892 0.700 16.005
 Method mAP (IoU=0.5) mAP (IoU=0.75) Inference time(s/im) YOLOv3 0.620 0.354 3.310 Faster R-CNN 0.741 0.439 13.254 Mask R-CNN 0.752 0.422 13.310 Big Map R-CNN 0.892 0.700 16.005
 [1] Wenmeng Geng, Kai Tao. Large deviation theorems for dirichlet determinants of analytic quasi-periodic jacobi operators with Brjuno-Rüssmann frequency. Communications on Pure & Applied Analysis, 2020, 19 (12) : 5305-5335. doi: 10.3934/cpaa.2020240 [2] Dandan Cheng, Qian Hao, Zhiming Li. Scale pressure for amenable group actions. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2021008 [3] Pablo D. Carrasco, Túlio Vales. A symmetric Random Walk defined by the time-one map of a geodesic flow. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020390 [4] Stefan Siegmund, Petr Stehlík. Time scale-induced asynchronous discrete dynamical systems. Discrete & Continuous Dynamical Systems - B, 2021, 26 (2) : 1011-1029. doi: 10.3934/dcdsb.2020151 [5] Mehdi Bastani, Davod Khojasteh Salkuyeh. On the GSOR iteration method for image restoration. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 27-43. doi: 10.3934/naco.2020013 [6] Qi Li, Hong Xue, Changxin Lu. Event-based fault detection for interval type-2 fuzzy systems with measurement outliers. Discrete & Continuous Dynamical Systems - S, 2021, 14 (4) : 1301-1328. doi: 10.3934/dcdss.2020412 [7] Manxue You, Shengjie Li. Perturbation of Image and conjugate duality for vector optimization. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020176 [8] Kateřina Škardová, Tomáš Oberhuber, Jaroslav Tintěra, Radomír Chabiniok. Signed-distance function based non-rigid registration of image series with varying image intensity. Discrete & Continuous Dynamical Systems - S, 2021, 14 (3) : 1145-1160. doi: 10.3934/dcdss.2020386 [9] Simone Fiori. Error-based control systems on Riemannian state manifolds: Properties of the principal pushforward map associated to parallel transport. Mathematical Control & Related Fields, 2021, 11 (1) : 143-167. doi: 10.3934/mcrf.2020031 [10] Longxiang Fang, Narayanaswamy Balakrishnan, Wenyu Huang. Stochastic comparisons of parallel systems with scale proportional hazards components equipped with starting devices. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2021004 [11] Abdelghafour Atlas, Mostafa Bendahmane, Fahd Karami, Driss Meskine, Omar Oubbih. A nonlinear fractional reaction-diffusion system applied to image denoising and decomposition. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020321 [12] Liam Burrows, Weihong Guo, Ke Chen, Francesco Torella. Reproducible kernel Hilbert space based global and local image segmentation. Inverse Problems & Imaging, 2021, 15 (1) : 1-25. doi: 10.3934/ipi.2020048 [13] Balázs Kósa, Karol Mikula, Markjoe Olunna Uba, Antonia Weberling, Neophytos Christodoulou, Magdalena Zernicka-Goetz. 3D image segmentation supported by a point cloud. Discrete & Continuous Dynamical Systems - S, 2021, 14 (3) : 971-985. doi: 10.3934/dcdss.2020351 [14] Emre Esentürk, Juan Velazquez. Large time behavior of exchange-driven growth. Discrete & Continuous Dynamical Systems - A, 2021, 41 (2) : 747-775. doi: 10.3934/dcds.2020299 [15] Huyuan Chen, Dong Ye, Feng Zhou. On gaussian curvature equation in $\mathbb{R}^2$ with prescribed nonpositive curvature. Discrete & Continuous Dynamical Systems - A, 2020, 40 (6) : 3201-3214. doi: 10.3934/dcds.2020125 [16] Andreu Ferré Moragues. Properties of multicorrelation sequences and large returns under some ergodicity assumptions. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020386 [17] Stefan Ruschel, Serhiy Yanchuk. The spectrum of delay differential equations with multiple hierarchical large delays. Discrete & Continuous Dynamical Systems - S, 2021, 14 (1) : 151-175. doi: 10.3934/dcdss.2020321 [18] Junyong Eom, Kazuhiro Ishige. Large time behavior of ODE type solutions to nonlinear diffusion equations. Discrete & Continuous Dynamical Systems - A, 2020, 40 (6) : 3395-3409. doi: 10.3934/dcds.2019229 [19] Tinghua Hu, Yang Yang, Zhengchun Zhou. Golay complementary sets with large zero odd-periodic correlation zones. Advances in Mathematics of Communications, 2021, 15 (1) : 23-33. doi: 10.3934/amc.2020040 [20] Xiaoping Zhai, Yongsheng Li. Global large solutions and optimal time-decay estimates to the Korteweg system. Discrete & Continuous Dynamical Systems - A, 2021, 41 (3) : 1387-1413. doi: 10.3934/dcds.2020322

Impact Factor: