Monocular Semantic Simultaneous Localization And Mapping

In recent years,visual Simultaneous Localization and Mapping(SLAM)has attracted more and more researchers to focus on,with the rapid development of computer vision.In this paper,we present a method for Simultaneous Localization and Mapping based on monocular semantic information,which is aimed to evaluate the localization of monocular visual motion sensors as well as the semantic map of their surrounding environment.A method for 3D object detection,which does not rely on the 3D model of the object that is prepared to be detected is presented so that the front-end of the SLAM system is able to fetch semantic information from the scene where motion sensors travel.This approach generates 3D proposal bounding boxes of objects from the result of 2D object detection,and then sifts the 3D proposal bounding boxes to get the final results.Thus,reprojection errors in the level of objects is presented,and the cost function,which is designed according to the reprojection errors mentioned above,is ready to be applied in the process of optimization afterwards.Moreover,this kind of reprojection error is also used to solve the problem of data association,which means that the combination that makes the total reprojection error smallest is the one that finally chose.In order to fetch geometric information from the scene,a method for feature point extraction is utilized to the front-end of the SLAM system.This method can robustly extract a great amount of low-level geometric information about the surrounding environment such as edges and angles.In the back-end of the SLAM system,geometric information,semantic information,and pose of motion sensors are jointly optimized under factor graph,which is a type of the framework of optimization,in order that the evaluation of semantic information and pose of motion sensors is more precise.In this algorithm,the semantic information is represented as 3D bounding boxes in the framework of factor graph optimization,and then the corresponding factors and vertices are added to the optimization process of joint optimization.In order to verify the effectiveness and practicality of the algorithm,the 3D object detection algorithm and the SLAM system proposed in this paper is tested and verified with the public dataset,Technische Universit?t München(TUM),and compared with several approaches.The result of the experiment shows that the 3D object detection algorithm is able to evaluate the pose and size of the target object more precisely,compared with Cube SLAM.Besides,the Absolute Trajectory Error(ATE)of the semantic SLAM system is less than ORBSLAM2 in partial scene(freiburg3_teddy,freiburg2_dishes and so on).