Research On Monocular Visual Semantic SLAM In Complex Environment

Traditional visual SLAM can effectively estimate pose and build environmental map based on geometric features such as point,line and plane,but this kind of map does not contain high-level semantic information of the environment.In order to make the robot understand the environment better,complete autonomous navigation more reasonably and perform more advanced Human-Robot Interaction(HRI)tasks,a monocular semantic SLAM system based on convolutional neural network(CNN)is proposed and implemented.Firstly,the theoretical basis of visual SLAM front-end is introduced from several aspects,such as pinhole camera model,binocular camera model,feature matching,epipolar geometry constraint,triangulation and PnP pose solving.Then the convolutional neural network is introduced from the network structure,loss function and normalization,which provides theoretical guidance for the implementation of monocular semantic SLAM system based on convolutional neural network.Secondly,in order to solve the problem of scale ambiguity in monocular SLAM,a scale-aware monocular SLAM system is proposed,which combines the traditional monocular SLAM with the monocular image depth estimation algorithm based on convolutional neural network,so as to significantly improve the accuracy of pose estimation and the robustness of monocular SLAM in pure rotation motion.Finally,on the basis of solving the scale uncertainty problem of monocular SLAM,a monocular semantic SLAM system based on convolutional neural network is designed.The image semantic segmentation algorithm based on convolutional neural network is designed in ROS environment,and the sparse map constructed by single ORB-SLAM is densified.Finally,the semantic segmented image is projected to the point-cloud map to complete the establishment of the semantic map.In order to verify the availability of the scale-aware monocular semantic SLAM algorithm,a series of experiments were carried out on the KITTI dataset,and the experimental results were analyzed qualitatively and quantitatively.The experimental results show that the proposed monocular semantic SLAM system can effectively improve the modeling and understanding ability of the robot in complex environment.