Research On Loop Closure Detection And Relocalization Based On The Fusion Of Vision And Lidar Data

SLAM-based mobile mapping technology has played an important role in the construction of a digital twin model in a working environment.As for complex working environments with the disturbance of dynamic objects and illumination changing,we propose a loop closure detection method based on image data and a loop closure detection method fusion of lidar and image data.On this basis,we integrate these methods into current SLAM systems and realize localization,mapping,and relocalization functions in a working environment.As for the visual-based loop closure detection problem in a dynamic environment,we propose a loop closure detection method based on image inpainting and image quality assessment.Firstly,the image semantic segmentation method is utilized to get dynamic regions.Secondly,the image inpainting method is used to inpainte dynamic regions as a corresponding static environment.Finally,feature points from bad inpainted quality areas are deleted through image quality assessment,which results in the improvement of visual-based loop closure detection in a dynamic environment.As for lidar-based loop closure detection problems in a dynamic environment,we propose loop closure detection and relocalization method based on semantic segmentation and objects matching.Firstly,the point clouds semantic segmentation method is used to get static point clouds and deletes connection point clouds between different objects to prevent these point clouds influence object-level point clouds segmentation.Based on semantic segmentation,the cluster-based method is utilized to realize point clouds instance segmentation for point clouds belonging to static objects.Secondly,global descriptors are extracted from point clouds without dynamic objects.Based on global descriptors matching,the candidate frame is detected.Finally,object descriptors fusion of point clouds and images are extracted and utilized to objects matching between two frames.The coarse registration results are got by objects matching based on geometric constraints and help to improve the precision of fine registration.Finally,the visual-based loop closure detection method is embedded into the ORBSLAM2 system,and the lidar-based loop closure detection method is embedded into the Le GOLOAM system.Especially,for the lidar and visual-based SLAM system,we realize relocalization,mapping,and refresh map after the relocalization function.We compare the SLAM system and current SLAM system in public outdoor environment KITTI dataset and our school dataset,which shows the efficiency of our SLAM system.