3D Scene Location And Reconstruction Based On Semantic Segmentation And Point Cloud Fusion

SLAM(Simultaneous Localization and Mapping)is a method in which a robot obtains its pose information and builds a point cloud map of the environment in an unknown environment.SLAM is a commonly used method of 3D reconstruction,which has broad application prospects and far-reaching research significance in the fields of mobile robots and autonomous driving.The three-dimensional point cloud map generated by the traditional SLAM algorithm can only reflect the real appearance of three-dimensional objects,does not contain the semantic information of the objects,and cannot perceive the types of objects in the environment.In order to solve the problems of robots or computers "understanding" the surrounding scenes,perceiving the environment anthropomorphically,and executing related decisions,the laser SLAM and visual SLAM technologies are used as the basis to estimate the position and construct 3D point cloud maps,respectively.The 3D point cloud map is then fused with the semantic segmentation information based on deep learning to reconstruct a3 D semantic map of the scene.Firstly,the related technologies are studied in domestic and international literature,the SLAM mathematical model is analyzed,and we study the basic theories and methods of laser SLAM and visual SLAM 3D scene reconstruction techniques and image semantic segmentation..Secondly,for the 3D semantic reconstruction problem of fusing point cloud maps generated by laser SLAM with semantic segmentation,the KITTI dataset is used as the simulation object.The combination of Le GO-LOAM(Lightweight and Ground Optimized LOAM)algorithm and semantic segmentation is used to complete the screening of the point cloud data provided by the KITTI dataset.The corresponding smoothing degree is calculated to obtain the 3D point cloud edge features and plane features,construct the feature constraint equations,calculate the LIDAR poses,and then use the pose-map optimization method to build the 3D point cloud map.The simulation is verified on KITTI dataset to realize 3D semantic reconstruction by fusing laser SLAM point cloud data with semantic segmentation information.Finally,for the 3D semantic reconstruction problem of fusing point cloud maps generated by visual SLAM with semantic segmentation,the kinect 1.0 camera is used as the input sensor and the laboratory is used as the experimental scenario.The ORBSLAM2 algorithm combined with semantic segmentation is used to extract ORB features on the image,and the features are matched to find the camera pose.A sparse map of the laboratory scene is established according to the key frame filtering criteria,an octree map is generated,a dense point cloud is estimated with the TSDF model,a semantic point cloud is generated by fusing the dense point cloud with semantic information,and then a 3D semantic map of the laboratory scene is obtained by fusing the semantic point cloud with the octree map,and the experimental results show that the 3D semantic reconstruction with the fusion of visual SLAM point cloud data and semantic segmentation information is achieved.