| The mobile robot SLAM(Simultaneous Localization and Mapping)technology is a key research content in the field of robotics.When working in a complex environment,the disadvantages of a single sensor will be magnified.However,multiple sensors,such as cameras,3D lidar and IMUs,can complement each other’s strengths and weaknesses.Therefore,this paper takes the mobile robot as the research object,integrates the advantages of various sensors,studies and implements the multi-sensor fusion SLAM system of the mobile robot,which includes three parts:extrinsic calibration between multiple sensors,fusion map construction and autonomous localization,etc.The specific content of the paper is as follows:1)Research on 3D lidar-camera-IMU calibration algorithm.This paper designed a multi-sensor suite was,and achieved hardware-level time synchronization.Then directly calibrated the extrinsic of 3D Lidar,camera and IMU.On the basis of directly method,the extrinsic between sensors were indirectly calculated by coordinate system transformation.Then the 3D lidar-camera-IMU extrinsic calibration algorithm was realized.2)Research on 3D lidar-vision fusion map construction algorithm.For 3D lidar loop closure detection,a fan-shaped descriptor was calculated for the multi-frame spliced point cloud,matched with historical keyframes and outliers were removed to obtain the lidar loop closure frame.For vision loop closure detection,a deep learning method was used to extract feature points,calculate the depth of feature point using line-plane intersection method,construct bag-of-words vector,search historical keyframes and remove outliers to obtain vision loop closure frame.The ICP algorithm was used to calculate the relative pose of lidar loop closure,bundle adjustment was used to calculate he relative pose of vision,and finally the pose constraints of lidar and vision were added to the pose graph for nonlinear optimization to construct a globally consistent fusion map,the 3D lidarvision fusion map construction algorithm were realized.3)Research on multi-sensor fusion localization algorithm.Using the IMU data to predict the state of the system,and integrated the known map into the observation stage,using the current point cloud and the known map to construct the point plane residual,using the ESIKF(error state iterative Kalman filter)algorithm to solve the 3D lidar odometry,and updated the known map.Aiming at the problem that single sensor relocation is easy to fail,a relocation algorithm based on 3D lidar and vision was designed.On this basis,a multi-sensor fusion localization system was constructed.The system can output both high-frequency odometry and robust lowfrequency global localization.Finally,relevant experimental research was carried out,and the experimental results verified the reliability and effectiveness of the system. |
