
-
Previous Article
Eliminating other-race effect for multi-ethnic facial expression recognition
- MFC Home
- This Issue
-
Next Article
Comparisons of different methods for balanced data classification under the discrete non-local total variational framework
SEMANTIC-RTAB-MAP (SRM): A semantic SLAM system with CNNs on depth images
1. | Beihang University, Beijing, China |
2. | Shenzhen Academy of Aerospace Technology, Shenzhen, China |
SLAM (simultaneous localization and mapping) system can be implemented based on monocular, RGB-D and stereo cameras. RTAB-MAP is a SLAM system, which can build dense 3D map. In this paper, we present a novel method named SEMANTIC-RTAB-MAP (SRM) to implement a semantic SLAM system based on RTAB-MAP and deep learning. We use YOLOv2 network to detect target objects in 2D images, and then use depth information for precise localization of the targets and finally add semantic information into 3D point clouds. We apply SRM in different scenes, and the results show its higher running speed and accuracy.
References:
[1] |
R. Q. Charles, H. Su, K. Mo and L. J. Guibas, Pointnet: Deep learning on point sets for 3d classification and segmentation, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2017).
doi: 10.1109/CVPR.2017.16. |
[2] |
R. Girshick and J. Donahue, Trevor Darrell and Jitendra Malik, Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation, (2013), 580-587. Google Scholar |
[3] |
K. He, X. Zhang, S. Ren and J. Sun, Deep residual learning for image recognition, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2016), 770-778.
doi: 10.1109/CVPR.2016.90. |
[4] |
M. Labbé and F. Michaud, Long-term online multi-session graph-based splam with memory management, Autonomous Robots, 3 (2017), 1-18. Google Scholar |
[5] |
M. Labbe and F. Michaud, Online global loop closure detection for large-scale multi-session graph-based SLAM, IEEE/RSJ International Conference on Intelligent Robots and Systems, (2014), 2661-2666.
doi: 10.1109/IROS.2014.6942926. |
[6] |
M. Labbé and F. Michaud, Appearance-based loop closure detection for online large-scale and long-term operation, IEEE Transactions on Robotics, 29 (2013), 734-745. Google Scholar |
[7] |
M. Labbe and F. Michaud, Memory management for real-time appearance-based loop closure detection, IEEE/RSJ International Conference on Intelligent Robots and Systems, (2011), 1271-1276.
doi: 10.1109/IROS.2011.6094602. |
[8] |
X. Li and R. Belaroussi, Semi-dense 3d semantic mapping from monocular slam, 2016. Google Scholar |
[9] |
W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed and C. Y. Fu, et al, SSD: Single Shot MultiBox Detector. European Conference on Computer Vision, Springer International Publishing, (2016), 21-37. Google Scholar |
[10] |
J. Mccormac, A. Handa, A. Davison and S. Leutenegger, Semanticfusion: dense 3d semantic mapping with convolutional neural networks, 2017 IEEE International Conference on Robotics and Automation (ICRA), 2017.
doi: 10.1109/ICRA.2017.7989538. |
[11] |
R. Mur-Artal and J. D. Tardós, Probabilistic semi-dense mapping from highly accurate feature-based monocular SLAM, Robotics: Science and Systems, (2015), 1-9.
doi: 10.15607/RSS.2015.XI.041. |
[12] |
N. Otsu,
A threshold selection method from gray-level histograms, IEEE Transactions on Systems, Man, and Cybernetics, 9 (1979), 62-66.
doi: 10.1109/TSMC.1979.4310076. |
[13] |
J. Redmon, S. Divvala, R. Girshick and A. Farhadi, You only look once: Unified, real-time object detection, Computer Vision and Pattern Recognition, (2016), 779-788.
doi: 10.1109/CVPR.2016.91. |
[14] |
J. Redmon and A. Farhadi, YOLO9000: Better, faster, stronger, IEEE Conference on Computer Vision and Pattern Recognition, (2017), 6517-6525.
doi: 10.1109/CVPR.2017.690. |
[15] |
J. Redmon and A. Farhadi, Yolov3: an incremental improvement, 2018. Google Scholar |
[16] |
S. Ren, K. He, R. Girshick and J. Sun,
Faster R-CNN: Towards real-time object detection with region proposal networks, IEEE Transactions on Pattern Analysis and Machine Intelligence, 39 (2017), 1137-1149.
doi: 10.1109/TPAMI.2016.2577031. |
[17] |
N. Sünderhauf, T. T. Pham, Y. Latif, M. Milford and I. Reid, Meaningful maps with object-oriented semantic mapping., Ieee/rsj International Conference on Intelligent Robots and Systems, IEEE, (2017), 5079-5085. Google Scholar |
[18] |
T. Whelan, S. Leutenegger, R. S. Moreno, B. Glocker and A. Davison, ElasticFusion: Dense SLAM Without A Pose Graph. Robotics: Science and Systems, 2015.
doi: 10.15607/RSS.2015.XI.001. |
show all references
References:
[1] |
R. Q. Charles, H. Su, K. Mo and L. J. Guibas, Pointnet: Deep learning on point sets for 3d classification and segmentation, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2017).
doi: 10.1109/CVPR.2017.16. |
[2] |
R. Girshick and J. Donahue, Trevor Darrell and Jitendra Malik, Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation, (2013), 580-587. Google Scholar |
[3] |
K. He, X. Zhang, S. Ren and J. Sun, Deep residual learning for image recognition, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2016), 770-778.
doi: 10.1109/CVPR.2016.90. |
[4] |
M. Labbé and F. Michaud, Long-term online multi-session graph-based splam with memory management, Autonomous Robots, 3 (2017), 1-18. Google Scholar |
[5] |
M. Labbe and F. Michaud, Online global loop closure detection for large-scale multi-session graph-based SLAM, IEEE/RSJ International Conference on Intelligent Robots and Systems, (2014), 2661-2666.
doi: 10.1109/IROS.2014.6942926. |
[6] |
M. Labbé and F. Michaud, Appearance-based loop closure detection for online large-scale and long-term operation, IEEE Transactions on Robotics, 29 (2013), 734-745. Google Scholar |
[7] |
M. Labbe and F. Michaud, Memory management for real-time appearance-based loop closure detection, IEEE/RSJ International Conference on Intelligent Robots and Systems, (2011), 1271-1276.
doi: 10.1109/IROS.2011.6094602. |
[8] |
X. Li and R. Belaroussi, Semi-dense 3d semantic mapping from monocular slam, 2016. Google Scholar |
[9] |
W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed and C. Y. Fu, et al, SSD: Single Shot MultiBox Detector. European Conference on Computer Vision, Springer International Publishing, (2016), 21-37. Google Scholar |
[10] |
J. Mccormac, A. Handa, A. Davison and S. Leutenegger, Semanticfusion: dense 3d semantic mapping with convolutional neural networks, 2017 IEEE International Conference on Robotics and Automation (ICRA), 2017.
doi: 10.1109/ICRA.2017.7989538. |
[11] |
R. Mur-Artal and J. D. Tardós, Probabilistic semi-dense mapping from highly accurate feature-based monocular SLAM, Robotics: Science and Systems, (2015), 1-9.
doi: 10.15607/RSS.2015.XI.041. |
[12] |
N. Otsu,
A threshold selection method from gray-level histograms, IEEE Transactions on Systems, Man, and Cybernetics, 9 (1979), 62-66.
doi: 10.1109/TSMC.1979.4310076. |
[13] |
J. Redmon, S. Divvala, R. Girshick and A. Farhadi, You only look once: Unified, real-time object detection, Computer Vision and Pattern Recognition, (2016), 779-788.
doi: 10.1109/CVPR.2016.91. |
[14] |
J. Redmon and A. Farhadi, YOLO9000: Better, faster, stronger, IEEE Conference on Computer Vision and Pattern Recognition, (2017), 6517-6525.
doi: 10.1109/CVPR.2017.690. |
[15] |
J. Redmon and A. Farhadi, Yolov3: an incremental improvement, 2018. Google Scholar |
[16] |
S. Ren, K. He, R. Girshick and J. Sun,
Faster R-CNN: Towards real-time object detection with region proposal networks, IEEE Transactions on Pattern Analysis and Machine Intelligence, 39 (2017), 1137-1149.
doi: 10.1109/TPAMI.2016.2577031. |
[17] |
N. Sünderhauf, T. T. Pham, Y. Latif, M. Milford and I. Reid, Meaningful maps with object-oriented semantic mapping., Ieee/rsj International Conference on Intelligent Robots and Systems, IEEE, (2017), 5079-5085. Google Scholar |
[18] |
T. Whelan, S. Leutenegger, R. S. Moreno, B. Glocker and A. Davison, ElasticFusion: Dense SLAM Without A Pose Graph. Robotics: Science and Systems, 2015.
doi: 10.15607/RSS.2015.XI.001. |












[1] |
José Madrid, João P. G. Ramos. On optimal autocorrelation inequalities on the real line. Communications on Pure & Applied Analysis, 2021, 20 (1) : 369-388. doi: 10.3934/cpaa.2020271 |
[2] |
Mehdi Bastani, Davod Khojasteh Salkuyeh. On the GSOR iteration method for image restoration. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 27-43. doi: 10.3934/naco.2020013 |
[3] |
Jing Zhou, Cheng Lu, Ye Tian, Xiaoying Tang. A SOCP relaxation based branch-and-bound method for generalized trust-region subproblem. Journal of Industrial & Management Optimization, 2021, 17 (1) : 151-168. doi: 10.3934/jimo.2019104 |
[4] |
Noah Stevenson, Ian Tice. A truncated real interpolation method and characterizations of screened Sobolev spaces. Communications on Pure & Applied Analysis, 2020, 19 (12) : 5509-5566. doi: 10.3934/cpaa.2020250 |
[5] |
Djamel Aaid, Amel Noui, Özen Özer. Piecewise quadratic bounding functions for finding real roots of polynomials. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 63-73. doi: 10.3934/naco.2020015 |
[6] |
Joan Carles Tatjer, Arturo Vieiro. Dynamics of the QR-flow for upper Hessenberg real matrices. Discrete & Continuous Dynamical Systems - B, 2021, 26 (3) : 1359-1403. doi: 10.3934/dcdsb.2020166 |
[7] |
Manxue You, Shengjie Li. Perturbation of Image and conjugate duality for vector optimization. Journal of Industrial & Management Optimization, 2020 doi: 10.3934/jimo.2020176 |
[8] |
Kateřina Škardová, Tomáš Oberhuber, Jaroslav Tintěra, Radomír Chabiniok. Signed-distance function based non-rigid registration of image series with varying image intensity. Discrete & Continuous Dynamical Systems - S, 2021, 14 (3) : 1145-1160. doi: 10.3934/dcdss.2020386 |
[9] |
Abdelghafour Atlas, Mostafa Bendahmane, Fahd Karami, Driss Meskine, Omar Oubbih. A nonlinear fractional reaction-diffusion system applied to image denoising and decomposition. Discrete & Continuous Dynamical Systems - B, 2020 doi: 10.3934/dcdsb.2020321 |
[10] |
Liam Burrows, Weihong Guo, Ke Chen, Francesco Torella. Reproducible kernel Hilbert space based global and local image segmentation. Inverse Problems & Imaging, 2021, 15 (1) : 1-25. doi: 10.3934/ipi.2020048 |
[11] |
Balázs Kósa, Karol Mikula, Markjoe Olunna Uba, Antonia Weberling, Neophytos Christodoulou, Magdalena Zernicka-Goetz. 3D image segmentation supported by a point cloud. Discrete & Continuous Dynamical Systems - S, 2021, 14 (3) : 971-985. doi: 10.3934/dcdss.2020351 |
[12] |
C. Burgos, J.-C. Cortés, L. Shaikhet, R.-J. Villanueva. A delayed nonlinear stochastic model for cocaine consumption: Stability analysis and simulation using real data. Discrete & Continuous Dynamical Systems - S, 2021, 14 (4) : 1233-1244. doi: 10.3934/dcdss.2020356 |
[13] |
Jia Cai, Guanglong Xu, Zhensheng Hu. Sketch-based image retrieval via CAT loss with elastic net regularization. Mathematical Foundations of Computing, 2020, 3 (4) : 219-227. doi: 10.3934/mfc.2020013 |
[14] |
Editorial Office. Retraction: Xiaohong Zhu, Zili Yang and Tabharit Zoubir, Research on the matching algorithm for heterologous image after deformation in the same scene. Discrete & Continuous Dynamical Systems - S, 2019, 12 (4&5) : 1281-1281. doi: 10.3934/dcdss.2019088 |
[15] |
Maika Goto, Kazunori Kuwana, Yasuhide Uegata, Shigetoshi Yazaki. A method how to determine parameters arising in a smoldering evolution equation by image segmentation for experiment's movies. Discrete & Continuous Dynamical Systems - S, 2021, 14 (3) : 881-891. doi: 10.3934/dcdss.2020233 |
[16] |
Sören Bartels, Jakob Keck. Adaptive time stepping in elastoplasticity. Discrete & Continuous Dynamical Systems - S, 2021, 14 (1) : 71-88. doi: 10.3934/dcdss.2020323 |
[17] |
Editorial Office. Retraction: Xiaohong Zhu, Lihe Zhou, Zili Yang and Joyati Debnath, A new text information extraction algorithm of video image under multimedia environment. Discrete & Continuous Dynamical Systems - S, 2019, 12 (4&5) : 1265-1265. doi: 10.3934/dcdss.2019087 |
[18] |
Simone Göttlich, Elisa Iacomini, Thomas Jung. Properties of the LWR model with time delay. Networks & Heterogeneous Media, 2020 doi: 10.3934/nhm.2020032 |
[19] |
Emre Esentürk, Juan Velazquez. Large time behavior of exchange-driven growth. Discrete & Continuous Dynamical Systems - A, 2021, 41 (2) : 747-775. doi: 10.3934/dcds.2020299 |
[20] |
Xu Zhang, Chuang Zheng, Enrique Zuazua. Time discrete wave equations: Boundary observability and control. Discrete & Continuous Dynamical Systems - A, 2009, 23 (1&2) : 571-604. doi: 10.3934/dcds.2009.23.571 |
Impact Factor:
Tools
Metrics
Other articles
by authors
[Back to Top]