January  2021, 17(1): 427-445. doi: 10.3934/jimo.2019119

Calibration of a 3D laser rangefinder and a camera based on optimization solution

a. 

School of Control Science and Engineering, Dalian University of Technology, Dalian 116024, China

b. 

School of Mathematical Sciences, Dalian University of Technology, Dalian 116024, China

c. 

Information Center, Ministry of Water Resources of China, Beijing 100032, China

* Corresponding author: Lei Wang (Email: wanglei@dlut.edu.cn)

Received  March 2019 Revised  April 2019 Published  September 2019

The calibration of a 3D laser rangefinder (LRF) and a camera is a key technique in the field of computer vision and intelligent robots. This paper proposes a new method for the calibration of a 3D LRF and a camera based on optimization solution. The calibration is achieved by freely moving a checkerboard pattern in front of the camera and the 3D LRF. The images and the 3D point clouds of the checkerboard pattern in various poses are collected by the camera and the 3D LRF respectively. By using the images, the intrinsic parameters and the poses of the checkerboard pattern are obtained. Then, two kinds of geometric constraints, line-to-plane constraints and plane-to-plane constraints, are constructed to solve the extrinsic parameters by linear optimization. Finally, the intrinsic and extrinsic parameters are further refined by global optimization, and are used to compute the geometric mapping relationship between the 3D LRF and the camera. The proposed calibration method is evaluated with both synthetic data and real data. The experimental results show that the proposed calibration method is accurate and robust to noise.

Citation: Yi An, Bo Li, Lei Wang, Chao Zhang, Xiaoli Zhou. Calibration of a 3D laser rangefinder and a camera based on optimization solution. Journal of Industrial & Management Optimization, 2021, 17 (1) : 427-445. doi: 10.3934/jimo.2019119
References:
[1]

Y. I. Abdel-Aziz and H. M. Karara, Direct linear transformation into object space coordinates in close-range photogrammetry, in Proceedings of the Symposium on Close-Range Photogrammetry, (1971), 1–18. Google Scholar

[2]

J.-Y. Bouguet, Camera Calibration Toolbox for Matlab, 2003. Google Scholar

[3]

J. Briales and J. Gonzalez-Jimenez, A minimal solution for the calibration of a 2D laser-rangefinder and a camera based on scene corners, in Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, (2015), 1891–1896. doi: 10.1109/IROS.2015.7353625.  Google Scholar

[4]

J. Y. Fan, On the levenberg-marquardt methods for convex constrained nonlinear equations, Journal of Industrial and Management Optimization, 9 (2013), 227-241.  doi: 10.3934/jimo.2013.9.227.  Google Scholar

[5]

O. D. Faugeras and G. Toscani, Camera calibration for 3D computer vision, in Proceedings of International Workshop on Industrial Application of Machine Vision and Machine Intelligence, (1987), 240–247. Google Scholar

[6]

A. Geiger, F. Moosmann, Ö. Car and B. Schuster, Automatic camera and range sensor calibration using a single shot, in Proceedings of IEEE International Conference on Robotics and Automation, (2012), 3936–3943. doi: 10.1109/ICRA.2012.6224570.  Google Scholar

[7]

R. Gomez-Ojeda, J. Briales, E. Fernandez-Moral and J. Gonzalez-Jimenez, Extrinsic calibration of a 2D laser-rangefinder and a camera based on scene corners, in Proceedings of IEEE International Conference on Robotics and Automation, (2015), 3611–3616. doi: 10.1109/ICRA.2015.7139700.  Google Scholar

[8]

X. J. GongY. Lin and J. L. Liu, 3D LIDAR-camera extrinsic calibration using an arbitrary trihedron, Sensors, 13 (2013), 1902-1918.  doi: 10.3390/s130201902.  Google Scholar

[9]

J. Heikkila and O. Silven, A four-step camera calibration procedure with implicit image correction, in Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, (1997), 1106–1112. doi: 10.1109/CVPR.1997.609468.  Google Scholar

[10]

Z. Z. HuY. C. LiN. Li and B. Zhao, Extrinsic calibration of 2-D laser rangefinder and camera from single shot based on minimal solution, IEEE Transactions on Instrumentation and Measurement, 65 (2016), 915-929.  doi: 10.1109/TIM.2016.2518248.  Google Scholar

[11]

G.-M. Lee, J.-H. Lee and S.-Y. Park, Calibration of VLP-16 Lidar and multi-view cameras using a ball for 360 degree 3D color map acquisition, in Proceedings of IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, (2017), 64–69. Google Scholar

[12]

Z. W. Liu, D. M. Lu, W. X. Qian, G. H. Gu, K. Ren, J. Zhang and X. F. Kong, Calibration of a single-point laser range finder and a camera, Optical and Quantum Electronics, 50 (2018), 447. doi: 10.1007/s11082-018-1702-y.  Google Scholar

[13]

T. Melen, Geometrical modelling and calibration of video cameras for underwater navigation, Psychotherapeut, 60 (1994), 351-352.   Google Scholar

[14]

J. J. Moré, The Levenberg-Marquardt algorithm: Implementation and theory, LNumerical Analysis, Lecture Notes in Math., Springer, Berlin, 630 (1978), 105–116.  Google Scholar

[15]

H. Rushmeier, J. Gomes, F. Giordano, H. E. Shishiny, K. Magerlein and F. Bernardini, Design and use of an in-museum system for artifact capture, in Proceedings of Conference on Computer Vision and Pattern Recognition Workshop, (2003), p8. doi: 10.1109/CVPRW.2003.10005.  Google Scholar

[16]

A. R. F. Sergio, V. Fremont and P. Bonnifait, Extrinsic calibration between a multi-layer Lidar and a camera, in Proceedings of IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, (2008), 214–219. Google Scholar

[17]

R. Tsai, A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses, IEEE Journal on Robotics and Automation, 3 (1987), 323-344.  doi: 10.1109/JRA.1987.1087109.  Google Scholar

[18]

F. VasconcelosJ. P. Barreto and U. Nunes, A minimal solution for the extrinsic calibration of a camera and a laser-rangefinder, IEEE Transactions on Pattern Analysis and Machine Intelligence, 34 (2012), 2097-2107.  doi: 10.1109/TPAMI.2012.18.  Google Scholar

[19]

L. Wang, F. C. Wu and Z. Y. Hu, Multi-camera calibration with one-dimensional object under general motions, in Proceedings of IEEE International Conference on Computer Vision, (2007), 1–7. doi: 10.1109/ICCV.2007.4408994.  Google Scholar

[20]

X. H. Ying, G. W. Wang, X. Mei, S. Yang, J. P. Rong and H. B. Zha, A direct method for the extrinsic calibration of a camera and a line scan LIDAR, in Proceedings of IEEE International Conference Mechatronics and Automation, (2014), 571–576. doi: 10.1109/ICMA.2014.6885760.  Google Scholar

[21]

Q. Zhang and R. Pless, Extrinsic calibration of a camera and laser range finder (improves camera calibration), in Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, (2004), 2301–2306. Google Scholar

[22]

Z. Zhang, A flexible new technique for camera calibration, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22 (2000), 1330-1334.  doi: 10.1109/34.888718.  Google Scholar

[23]

Z. Y. Zhang, Camera calibration with one-dimensional objects, Computer Vision-ECCV 2002, (2002), 161–174. doi: 10.1007/3-540-47979-1_11.  Google Scholar

[24]

L. P. Zhou, A new minimal solution for the extrinsic calibration of a 2D LIDAR and a camera using three plane-line correspondences, IEEE Sensors Joural, 14 (2014), 442-454.  doi: 10.1109/JSEN.2013.2284789.  Google Scholar

[25]

Y. ZhuangF. Yan and H. S. Hu, Automatic extrinsic self-calibration for fusing data from monocular vision and 3-D laser scanner, IEEE Transactions on Instrumentation and Measurement, 63 (2014), 1874-1876.  doi: 10.1109/TIM.2014.2307731.  Google Scholar

show all references

References:
[1]

Y. I. Abdel-Aziz and H. M. Karara, Direct linear transformation into object space coordinates in close-range photogrammetry, in Proceedings of the Symposium on Close-Range Photogrammetry, (1971), 1–18. Google Scholar

[2]

J.-Y. Bouguet, Camera Calibration Toolbox for Matlab, 2003. Google Scholar

[3]

J. Briales and J. Gonzalez-Jimenez, A minimal solution for the calibration of a 2D laser-rangefinder and a camera based on scene corners, in Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, (2015), 1891–1896. doi: 10.1109/IROS.2015.7353625.  Google Scholar

[4]

J. Y. Fan, On the levenberg-marquardt methods for convex constrained nonlinear equations, Journal of Industrial and Management Optimization, 9 (2013), 227-241.  doi: 10.3934/jimo.2013.9.227.  Google Scholar

[5]

O. D. Faugeras and G. Toscani, Camera calibration for 3D computer vision, in Proceedings of International Workshop on Industrial Application of Machine Vision and Machine Intelligence, (1987), 240–247. Google Scholar

[6]

A. Geiger, F. Moosmann, Ö. Car and B. Schuster, Automatic camera and range sensor calibration using a single shot, in Proceedings of IEEE International Conference on Robotics and Automation, (2012), 3936–3943. doi: 10.1109/ICRA.2012.6224570.  Google Scholar

[7]

R. Gomez-Ojeda, J. Briales, E. Fernandez-Moral and J. Gonzalez-Jimenez, Extrinsic calibration of a 2D laser-rangefinder and a camera based on scene corners, in Proceedings of IEEE International Conference on Robotics and Automation, (2015), 3611–3616. doi: 10.1109/ICRA.2015.7139700.  Google Scholar

[8]

X. J. GongY. Lin and J. L. Liu, 3D LIDAR-camera extrinsic calibration using an arbitrary trihedron, Sensors, 13 (2013), 1902-1918.  doi: 10.3390/s130201902.  Google Scholar

[9]

J. Heikkila and O. Silven, A four-step camera calibration procedure with implicit image correction, in Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, (1997), 1106–1112. doi: 10.1109/CVPR.1997.609468.  Google Scholar

[10]

Z. Z. HuY. C. LiN. Li and B. Zhao, Extrinsic calibration of 2-D laser rangefinder and camera from single shot based on minimal solution, IEEE Transactions on Instrumentation and Measurement, 65 (2016), 915-929.  doi: 10.1109/TIM.2016.2518248.  Google Scholar

[11]

G.-M. Lee, J.-H. Lee and S.-Y. Park, Calibration of VLP-16 Lidar and multi-view cameras using a ball for 360 degree 3D color map acquisition, in Proceedings of IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, (2017), 64–69. Google Scholar

[12]

Z. W. Liu, D. M. Lu, W. X. Qian, G. H. Gu, K. Ren, J. Zhang and X. F. Kong, Calibration of a single-point laser range finder and a camera, Optical and Quantum Electronics, 50 (2018), 447. doi: 10.1007/s11082-018-1702-y.  Google Scholar

[13]

T. Melen, Geometrical modelling and calibration of video cameras for underwater navigation, Psychotherapeut, 60 (1994), 351-352.   Google Scholar

[14]

J. J. Moré, The Levenberg-Marquardt algorithm: Implementation and theory, LNumerical Analysis, Lecture Notes in Math., Springer, Berlin, 630 (1978), 105–116.  Google Scholar

[15]

H. Rushmeier, J. Gomes, F. Giordano, H. E. Shishiny, K. Magerlein and F. Bernardini, Design and use of an in-museum system for artifact capture, in Proceedings of Conference on Computer Vision and Pattern Recognition Workshop, (2003), p8. doi: 10.1109/CVPRW.2003.10005.  Google Scholar

[16]

A. R. F. Sergio, V. Fremont and P. Bonnifait, Extrinsic calibration between a multi-layer Lidar and a camera, in Proceedings of IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, (2008), 214–219. Google Scholar

[17]

R. Tsai, A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses, IEEE Journal on Robotics and Automation, 3 (1987), 323-344.  doi: 10.1109/JRA.1987.1087109.  Google Scholar

[18]

F. VasconcelosJ. P. Barreto and U. Nunes, A minimal solution for the extrinsic calibration of a camera and a laser-rangefinder, IEEE Transactions on Pattern Analysis and Machine Intelligence, 34 (2012), 2097-2107.  doi: 10.1109/TPAMI.2012.18.  Google Scholar

[19]

L. Wang, F. C. Wu and Z. Y. Hu, Multi-camera calibration with one-dimensional object under general motions, in Proceedings of IEEE International Conference on Computer Vision, (2007), 1–7. doi: 10.1109/ICCV.2007.4408994.  Google Scholar

[20]

X. H. Ying, G. W. Wang, X. Mei, S. Yang, J. P. Rong and H. B. Zha, A direct method for the extrinsic calibration of a camera and a line scan LIDAR, in Proceedings of IEEE International Conference Mechatronics and Automation, (2014), 571–576. doi: 10.1109/ICMA.2014.6885760.  Google Scholar

[21]

Q. Zhang and R. Pless, Extrinsic calibration of a camera and laser range finder (improves camera calibration), in Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, (2004), 2301–2306. Google Scholar

[22]

Z. Zhang, A flexible new technique for camera calibration, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22 (2000), 1330-1334.  doi: 10.1109/34.888718.  Google Scholar

[23]

Z. Y. Zhang, Camera calibration with one-dimensional objects, Computer Vision-ECCV 2002, (2002), 161–174. doi: 10.1007/3-540-47979-1_11.  Google Scholar

[24]

L. P. Zhou, A new minimal solution for the extrinsic calibration of a 2D LIDAR and a camera using three plane-line correspondences, IEEE Sensors Joural, 14 (2014), 442-454.  doi: 10.1109/JSEN.2013.2284789.  Google Scholar

[25]

Y. ZhuangF. Yan and H. S. Hu, Automatic extrinsic self-calibration for fusing data from monocular vision and 3-D laser scanner, IEEE Transactions on Instrumentation and Measurement, 63 (2014), 1874-1876.  doi: 10.1109/TIM.2014.2307731.  Google Scholar

Figure 1.  Calibration board and 3D color laser ranging system
Figure 2.  Pinhole camera model
Figure 3.  Point cloud of the calibration board and its line point clouds
Figure 4.  Calibration images
Figure 5.  Normal vector $ \bar{\mathit{\boldsymbol{n}}}_{j} $ of the calibration board in $ j $th pose
Figure 6.  Line-to-plane constraint
Figure 7.  The plane-to-plane constraint
Figure 8.  Synthetic data generation
Figure 9.  Calibration results with the increasing number of poses
Figure 10.  Calibration results with the increasing noise level
Figure 11.  Calibration results with the number of poses 30 and the noise level 10$ mm $
Figure 12.  3D color point cloud of the room
Figure 13.  3D color point cloud of the housing estate
Table 1.  RMS error analysis results (unit: pixel)
Method Minimum Average Maximum
Our Method 1.3571 3.6962 7.2579
Zhang's Method 1.8495 4.7702 8.6392
Ying's Method 1.5946 3.7866 7.4648
Method Minimum Average Maximum
Our Method 1.3571 3.6962 7.2579
Zhang's Method 1.8495 4.7702 8.6392
Ying's Method 1.5946 3.7866 7.4648
[1]

Luke Finlay, Vladimir Gaitsgory, Ivan Lebedev. Linear programming solutions of periodic optimization problems: approximation of the optimal control. Journal of Industrial & Management Optimization, 2007, 3 (2) : 399-413. doi: 10.3934/jimo.2007.3.399

[2]

Ziteng Wang, Shu-Cherng Fang, Wenxun Xing. On constraint qualifications: Motivation, design and inter-relations. Journal of Industrial & Management Optimization, 2013, 9 (4) : 983-1001. doi: 10.3934/jimo.2013.9.983

[3]

Fritz Gesztesy, Helge Holden, Johanna Michor, Gerald Teschl. The algebro-geometric initial value problem for the Ablowitz-Ladik hierarchy. Discrete & Continuous Dynamical Systems - A, 2010, 26 (1) : 151-196. doi: 10.3934/dcds.2010.26.151

[4]

Ardeshir Ahmadi, Hamed Davari-Ardakani. A multistage stochastic programming framework for cardinality constrained portfolio optimization. Numerical Algebra, Control & Optimization, 2017, 7 (3) : 359-377. doi: 10.3934/naco.2017023

[5]

Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437

[6]

Guillaume Bal, Wenjia Jing. Homogenization and corrector theory for linear transport in random media. Discrete & Continuous Dynamical Systems - A, 2010, 28 (4) : 1311-1343. doi: 10.3934/dcds.2010.28.1311

[7]

Nizami A. Gasilov. Solving a system of linear differential equations with interval coefficients. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2739-2747. doi: 10.3934/dcdsb.2020203

[8]

Hong Seng Sim, Wah June Leong, Chuei Yee Chen, Siti Nur Iqmal Ibrahim. Multi-step spectral gradient methods with modified weak secant relation for large scale unconstrained optimization. Numerical Algebra, Control & Optimization, 2018, 8 (3) : 377-387. doi: 10.3934/naco.2018024

[9]

Abdulrazzaq T. Abed, Azzam S. Y. Aladool. Applying particle swarm optimization based on Padé approximant to solve ordinary differential equation. Numerical Algebra, Control & Optimization, 2021  doi: 10.3934/naco.2021008

[10]

Mohsen Abdolhosseinzadeh, Mir Mohammad Alipour. Design of experiment for tuning parameters of an ant colony optimization method for the constrained shortest Hamiltonian path problem in the grid networks. Numerical Algebra, Control & Optimization, 2021, 11 (2) : 321-332. doi: 10.3934/naco.2020028

[11]

Reza Lotfi, Yahia Zare Mehrjerdi, Mir Saman Pishvaee, Ahmad Sadeghieh, Gerhard-Wilhelm Weber. A robust optimization model for sustainable and resilient closed-loop supply chain network design considering conditional value at risk. Numerical Algebra, Control & Optimization, 2021, 11 (2) : 221-253. doi: 10.3934/naco.2020023

[12]

Alexander A. Davydov, Massimo Giulietti, Stefano Marcugini, Fernanda Pambianco. Linear nonbinary covering codes and saturating sets in projective spaces. Advances in Mathematics of Communications, 2011, 5 (1) : 119-147. doi: 10.3934/amc.2011.5.119

[13]

W. Cary Huffman. On the theory of $\mathbb{F}_q$-linear $\mathbb{F}_{q^t}$-codes. Advances in Mathematics of Communications, 2013, 7 (3) : 349-378. doi: 10.3934/amc.2013.7.349

[14]

Arunima Bhattacharya, Micah Warren. $ C^{2, \alpha} $ estimates for solutions to almost Linear elliptic equations. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2021024

[15]

Misha Bialy, Andrey E. Mironov. Rich quasi-linear system for integrable geodesic flows on 2-torus. Discrete & Continuous Dynamical Systems - A, 2011, 29 (1) : 81-90. doi: 10.3934/dcds.2011.29.81

[16]

Fumihiko Nakamura. Asymptotic behavior of non-expanding piecewise linear maps in the presence of random noise. Discrete & Continuous Dynamical Systems - B, 2018, 23 (6) : 2457-2473. doi: 10.3934/dcdsb.2018055

[17]

Hirofumi Notsu, Masato Kimura. Symmetry and positive definiteness of the tensor-valued spring constant derived from P1-FEM for the equations of linear elasticity. Networks & Heterogeneous Media, 2014, 9 (4) : 617-634. doi: 10.3934/nhm.2014.9.617

[18]

Jong Yoon Hyun, Yoonjin Lee, Yansheng Wu. Connection of $ p $-ary $ t $-weight linear codes to Ramanujan Cayley graphs with $ t+1 $ eigenvalues. Advances in Mathematics of Communications, 2021  doi: 10.3934/amc.2020133

2019 Impact Factor: 1.366

Metrics

  • PDF downloads (121)
  • HTML views (516)
  • Cited by (0)

Other articles
by authors

[Back to Top]