
-
Previous Article
Eliminating other-race effect for multi-ethnic facial expression recognition
- MFC Home
- This Issue
-
Next Article
Comparisons of different methods for balanced data classification under the discrete non-local total variational framework
SEMANTIC-RTAB-MAP (SRM): A semantic SLAM system with CNNs on depth images
1. | Beihang University, Beijing, China |
2. | Shenzhen Academy of Aerospace Technology, Shenzhen, China |
SLAM (simultaneous localization and mapping) system can be implemented based on monocular, RGB-D and stereo cameras. RTAB-MAP is a SLAM system, which can build dense 3D map. In this paper, we present a novel method named SEMANTIC-RTAB-MAP (SRM) to implement a semantic SLAM system based on RTAB-MAP and deep learning. We use YOLOv2 network to detect target objects in 2D images, and then use depth information for precise localization of the targets and finally add semantic information into 3D point clouds. We apply SRM in different scenes, and the results show its higher running speed and accuracy.
References:
[1] |
R. Q. Charles, H. Su, K. Mo and L. J. Guibas, Pointnet: Deep learning on point sets for 3d classification and segmentation, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2017).
doi: 10.1109/CVPR.2017.16. |
[2] |
R. Girshick and J. Donahue, Trevor Darrell and Jitendra Malik, Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation, (2013), 580-587. Google Scholar |
[3] |
K. He, X. Zhang, S. Ren and J. Sun, Deep residual learning for image recognition, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2016), 770-778.
doi: 10.1109/CVPR.2016.90. |
[4] |
M. Labbé and F. Michaud, Long-term online multi-session graph-based splam with memory management, Autonomous Robots, 3 (2017), 1-18. Google Scholar |
[5] |
M. Labbe and F. Michaud, Online global loop closure detection for large-scale multi-session graph-based SLAM, IEEE/RSJ International Conference on Intelligent Robots and Systems, (2014), 2661-2666.
doi: 10.1109/IROS.2014.6942926. |
[6] |
M. Labbé and F. Michaud, Appearance-based loop closure detection for online large-scale and long-term operation, IEEE Transactions on Robotics, 29 (2013), 734-745. Google Scholar |
[7] |
M. Labbe and F. Michaud, Memory management for real-time appearance-based loop closure detection, IEEE/RSJ International Conference on Intelligent Robots and Systems, (2011), 1271-1276.
doi: 10.1109/IROS.2011.6094602. |
[8] |
X. Li and R. Belaroussi, Semi-dense 3d semantic mapping from monocular slam, 2016. Google Scholar |
[9] |
W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed and C. Y. Fu, et al, SSD: Single Shot MultiBox Detector. European Conference on Computer Vision, Springer International Publishing, (2016), 21-37. Google Scholar |
[10] |
J. Mccormac, A. Handa, A. Davison and S. Leutenegger, Semanticfusion: dense 3d semantic mapping with convolutional neural networks, 2017 IEEE International Conference on Robotics and Automation (ICRA), 2017.
doi: 10.1109/ICRA.2017.7989538. |
[11] |
R. Mur-Artal and J. D. Tardós, Probabilistic semi-dense mapping from highly accurate feature-based monocular SLAM, Robotics: Science and Systems, (2015), 1-9.
doi: 10.15607/RSS.2015.XI.041. |
[12] |
N. Otsu,
A threshold selection method from gray-level histograms, IEEE Transactions on Systems, Man, and Cybernetics, 9 (1979), 62-66.
doi: 10.1109/TSMC.1979.4310076. |
[13] |
J. Redmon, S. Divvala, R. Girshick and A. Farhadi, You only look once: Unified, real-time object detection, Computer Vision and Pattern Recognition, (2016), 779-788.
doi: 10.1109/CVPR.2016.91. |
[14] |
J. Redmon and A. Farhadi, YOLO9000: Better, faster, stronger, IEEE Conference on Computer Vision and Pattern Recognition, (2017), 6517-6525.
doi: 10.1109/CVPR.2017.690. |
[15] |
J. Redmon and A. Farhadi, Yolov3: an incremental improvement, 2018. Google Scholar |
[16] |
S. Ren, K. He, R. Girshick and J. Sun,
Faster R-CNN: Towards real-time object detection with region proposal networks, IEEE Transactions on Pattern Analysis and Machine Intelligence, 39 (2017), 1137-1149.
doi: 10.1109/TPAMI.2016.2577031. |
[17] |
N. Sünderhauf, T. T. Pham, Y. Latif, M. Milford and I. Reid, Meaningful maps with object-oriented semantic mapping., Ieee/rsj International Conference on Intelligent Robots and Systems, IEEE, (2017), 5079-5085. Google Scholar |
[18] |
T. Whelan, S. Leutenegger, R. S. Moreno, B. Glocker and A. Davison, ElasticFusion: Dense SLAM Without A Pose Graph. Robotics: Science and Systems, 2015.
doi: 10.15607/RSS.2015.XI.001. |
show all references
References:
[1] |
R. Q. Charles, H. Su, K. Mo and L. J. Guibas, Pointnet: Deep learning on point sets for 3d classification and segmentation, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2017).
doi: 10.1109/CVPR.2017.16. |
[2] |
R. Girshick and J. Donahue, Trevor Darrell and Jitendra Malik, Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation, (2013), 580-587. Google Scholar |
[3] |
K. He, X. Zhang, S. Ren and J. Sun, Deep residual learning for image recognition, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2016), 770-778.
doi: 10.1109/CVPR.2016.90. |
[4] |
M. Labbé and F. Michaud, Long-term online multi-session graph-based splam with memory management, Autonomous Robots, 3 (2017), 1-18. Google Scholar |
[5] |
M. Labbe and F. Michaud, Online global loop closure detection for large-scale multi-session graph-based SLAM, IEEE/RSJ International Conference on Intelligent Robots and Systems, (2014), 2661-2666.
doi: 10.1109/IROS.2014.6942926. |
[6] |
M. Labbé and F. Michaud, Appearance-based loop closure detection for online large-scale and long-term operation, IEEE Transactions on Robotics, 29 (2013), 734-745. Google Scholar |
[7] |
M. Labbe and F. Michaud, Memory management for real-time appearance-based loop closure detection, IEEE/RSJ International Conference on Intelligent Robots and Systems, (2011), 1271-1276.
doi: 10.1109/IROS.2011.6094602. |
[8] |
X. Li and R. Belaroussi, Semi-dense 3d semantic mapping from monocular slam, 2016. Google Scholar |
[9] |
W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed and C. Y. Fu, et al, SSD: Single Shot MultiBox Detector. European Conference on Computer Vision, Springer International Publishing, (2016), 21-37. Google Scholar |
[10] |
J. Mccormac, A. Handa, A. Davison and S. Leutenegger, Semanticfusion: dense 3d semantic mapping with convolutional neural networks, 2017 IEEE International Conference on Robotics and Automation (ICRA), 2017.
doi: 10.1109/ICRA.2017.7989538. |
[11] |
R. Mur-Artal and J. D. Tardós, Probabilistic semi-dense mapping from highly accurate feature-based monocular SLAM, Robotics: Science and Systems, (2015), 1-9.
doi: 10.15607/RSS.2015.XI.041. |
[12] |
N. Otsu,
A threshold selection method from gray-level histograms, IEEE Transactions on Systems, Man, and Cybernetics, 9 (1979), 62-66.
doi: 10.1109/TSMC.1979.4310076. |
[13] |
J. Redmon, S. Divvala, R. Girshick and A. Farhadi, You only look once: Unified, real-time object detection, Computer Vision and Pattern Recognition, (2016), 779-788.
doi: 10.1109/CVPR.2016.91. |
[14] |
J. Redmon and A. Farhadi, YOLO9000: Better, faster, stronger, IEEE Conference on Computer Vision and Pattern Recognition, (2017), 6517-6525.
doi: 10.1109/CVPR.2017.690. |
[15] |
J. Redmon and A. Farhadi, Yolov3: an incremental improvement, 2018. Google Scholar |
[16] |
S. Ren, K. He, R. Girshick and J. Sun,
Faster R-CNN: Towards real-time object detection with region proposal networks, IEEE Transactions on Pattern Analysis and Machine Intelligence, 39 (2017), 1137-1149.
doi: 10.1109/TPAMI.2016.2577031. |
[17] |
N. Sünderhauf, T. T. Pham, Y. Latif, M. Milford and I. Reid, Meaningful maps with object-oriented semantic mapping., Ieee/rsj International Conference on Intelligent Robots and Systems, IEEE, (2017), 5079-5085. Google Scholar |
[18] |
T. Whelan, S. Leutenegger, R. S. Moreno, B. Glocker and A. Davison, ElasticFusion: Dense SLAM Without A Pose Graph. Robotics: Science and Systems, 2015.
doi: 10.15607/RSS.2015.XI.001. |












[1] |
Frank Sottile. The special Schubert calculus is real. Electronic Research Announcements, 1999, 5: 35-39. |
[2] |
Todd Hurst, Volker Rehbock. Optimizing micro-algae production in a raceway pond with variable depth. Journal of Industrial & Management Optimization, 2021 doi: 10.3934/jimo.2021027 |
[3] |
Lekbir Afraites, Abdelghafour Atlas, Fahd Karami, Driss Meskine. Some class of parabolic systems applied to image processing. Discrete & Continuous Dynamical Systems - B, 2016, 21 (6) : 1671-1687. doi: 10.3934/dcdsb.2016017 |
[4] |
Zhihua Zhang, Naoki Saito. PHLST with adaptive tiling and its application to antarctic remote sensing image approximation. Inverse Problems & Imaging, 2014, 8 (1) : 321-337. doi: 10.3934/ipi.2014.8.321 |
[5] |
Israa Mohammed Khudher, Yahya Ismail Ibrahim, Suhaib Abduljabbar Altamir. Individual biometrics pattern based artificial image analysis techniques. Numerical Algebra, Control & Optimization, 2021 doi: 10.3934/naco.2020056 |
[6] |
Jianping Gao, Shangjiang Guo, Wenxian Shen. Persistence and time periodic positive solutions of doubly nonlocal Fisher-KPP equations in time periodic and space heterogeneous media. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2645-2676. doi: 10.3934/dcdsb.2020199 |
[7] |
Jia Cai, Guanglong Xu, Zhensheng Hu. Sketch-based image retrieval via CAT loss with elastic net regularization. Mathematical Foundations of Computing, 2020, 3 (4) : 219-227. doi: 10.3934/mfc.2020013 |
[8] |
Cécile Carrère, Grégoire Nadin. Influence of mutations in phenotypically-structured populations in time periodic environment. Discrete & Continuous Dynamical Systems - B, 2020, 25 (9) : 3609-3630. doi: 10.3934/dcdsb.2020075 |
[9] |
Paula A. González-Parra, Sunmi Lee, Leticia Velázquez, Carlos Castillo-Chavez. A note on the use of optimal control on a discrete time model of influenza dynamics. Mathematical Biosciences & Engineering, 2011, 8 (1) : 183-197. doi: 10.3934/mbe.2011.8.183 |
[10] |
Guillermo Reyes, Juan-Luis Vázquez. Long time behavior for the inhomogeneous PME in a medium with slowly decaying density. Communications on Pure & Applied Analysis, 2009, 8 (2) : 493-508. doi: 10.3934/cpaa.2009.8.493 |
[11] |
Wei-Jian Bo, Guo Lin, Shigui Ruan. Traveling wave solutions for time periodic reaction-diffusion systems. Discrete & Continuous Dynamical Systems - A, 2018, 38 (9) : 4329-4351. doi: 10.3934/dcds.2018189 |
[12] |
Kin Ming Hui, Soojung Kim. Asymptotic large time behavior of singular solutions of the fast diffusion equation. Discrete & Continuous Dynamical Systems - A, 2017, 37 (11) : 5943-5977. doi: 10.3934/dcds.2017258 |
[13] |
Linlin Li, Bedreddine Ainseba. Large-time behavior of matured population in an age-structured model. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2561-2580. doi: 10.3934/dcdsb.2020195 |
[14] |
Tomáš Roubíček. An energy-conserving time-discretisation scheme for poroelastic media with phase-field fracture emitting waves and heat. Discrete & Continuous Dynamical Systems - S, 2017, 10 (4) : 867-893. doi: 10.3934/dcdss.2017044 |
[15] |
Xiaomao Deng, Xiao-Chuan Cai, Jun Zou. A parallel space-time domain decomposition method for unsteady source inversion problems. Inverse Problems & Imaging, 2015, 9 (4) : 1069-1091. doi: 10.3934/ipi.2015.9.1069 |
[16] |
Zengyun Wang, Jinde Cao, Zuowei Cai, Lihong Huang. Finite-time stability of impulsive differential inclusion: Applications to discontinuous impulsive neural networks. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2677-2692. doi: 10.3934/dcdsb.2020200 |
Impact Factor:
Tools
Article outline
Figures and Tables
[Back to Top]