Deep Learning Based Semantic Map Construction In Visual SLAM

In recent years,with the rapid development of science and technology,the research of intelligent robots has received more and more attentions.In the research of intelligent robots,Simultaneous Localization and Mapping(SLAM)is a fundamental problem that needs to be solved.Most of the current visual SLAM systems depend on detecting and matching visual feature points,which makes the robot SLAM system can only take advantages of the geometrical information in the environment,but not the rich semantic information in the environment.RGBD cameras can directly obtain the distance information of objects,robot vision research based on RGBD sensors has attracted the attention of many scholars.Generally,RGBD data can be used to reconstruct the 3D point cloud map of the environment.The point cloud map generally contains relatively rich spatial geometric information.However,point cloud data is usually irregular space discrete point,and it is difficult to obtain semantic information from it.Therefore,how to let the robot obtain the semantic information of the surrounding environment,which makes the robot has the perceptual ability,has become a research hotspot in visual SLAM.In this paper,we use ORB-SLAM2 system to construct dense point cloud map in RGBD image sequence,and use convolutional neural network to extract semantic information during the process of mapping,Then,a graph-based cutting algorithm is used to cut the point cloud map,Finally,the semantic information is projected to the point cloud cluster to realize the 3D dense semantic map construction.This paper verifies the effectiveness of the 3D dense semantic map construction algorithm on popular public datasets.The main work and contributions of this paper are summarized as follows:1.An efficient and fast point cloud cutting algorithm is proposed.The algorithm firstly uses supervoxel method to preprocess the original point cloud data,which reduces the complexity of subsequent calculations and makes the point set features more concentrated.Then,it combines the local cloud point space and the global plane information,re-cutting the pre-processed point cloud data by the graph-based cutting method,achieving better and faster cutting of the point cloud area.2.A semantic map construction method that utilizes both 2D image information and 3D point cloud information data is proposed.The image has rich texture and color information,and the 3D point cloud data also has rich spatial geometric information.There are few documents perform semantic analysis based on both of them.In this paper,we use the deep learning target detection algorithm to extract semantic information from RGB images of the environment,Then,the reconstructed point cloud is cut regionally.Finally,we fuse both of them to realize the point cloud semantic map construction.3.A semantic SLAM system based on ORB-SLAM2 is constructed.Based on ORB-SLAM2,this paper realizes the construction of dense point cloud semantic map by means of deep learning and point cloud cutting method,which has reference and guiding significance for enhancing the perception ability of robot.