Deep Semantic Simultaneous Localization And Mapping

Visual SLAM is helpful for the development of unmanned platform.But only perceiving the geometric information of the environment is not enough.It's necessary to join the image semantic segmentation with SLAM,which helps to comprehend the scenes and improve the accuracy of pose estimation.Nowadays,the technique of deep learning brings new ideas into traditional SLAMs by modeling this problem with convolution neural network.Though the results barely satisfactory,it draws the researchers' attention.Therefore,aiming at the practical needs and new challenges,this thesis proposes a model of deep semantic SLAM.This thesis begins with end-to-end pose learning network,learning pose transformation between image frames by CNN.Based on that,depth information is joined in by referring to the direct method in the traditional SLAMs,and the depth information is supervised by the sparse point cloud and make for photometric error with the predicted pose.Lastly the results of pose estimation equally matches the traditional SLAMs.Furthermore,semantic information is added to the network as one of the learning tasks.It weights each pixel and screen the effective pixels that have a positive effect on pose estimation.With the help of semantic information,the accuracy of pose estimation gets more improvement.In order to solve the problem of accumulated errors in long sequences,here we optimize the predicted poses by Pose Graph.Finally,the optimized results is combined with the depth and semantic information,resulting 3D maps and semantic maps of the scene.Our experiments on two publicly available datasets demonstrate our model improves the accuracy of pose estimation,which not only outperforms ORB-SLAM and other model based on deep learning,but also has robustness to some environment changes.