Optimization Research On Semantic Segmentation Fused High-precision Stereo Visual SLAM System

The visual-based simultaneous localization and mapping(SLAM)is one of the key technique for mobile robots to fulfilling localization and navigation in an unknown environment,and it is also the core intelligent research problem of mobile robots.The traditional visual SLAM technology is limited to the geometric features(feature points or lines)of the image and cannot establish semantic map of the environment for human-computer interaction,and also suffer in localization accuracy in the dynamic scene,So it is hard for widely use yet.The stereo visual SLAM system is currently the most accuracy and widely used visual SLAM system.Based on that,this paper proposed a semantic segmentation fused highprecision stereo visual SLAM system,which using semantic segmentation output of convolutional neural network and semantic fused optimization algorithm to optimize the components of the traditional stereo visual SLAM.The proposed system is fully verified on public dataset.Experiments show that compared with the traditional stereo visual SLAM,the proposed system has higher localization accuracy in dynamic scenes.In addition,the output semi-dense semantic map of the system is more benefit for human-computer interaction.This paper consist of the main work as follows:1)Model selection and transfer learning of convolutional neural networks.Fusing convolutional neural networks into SLAM frameworks.And proposed a real-time semantic segmentation method.2)Designed a faster and more accurate feature matching algorithm based on semantic segmentation information and dynamic object detection algorithm to improve the localization accuracy in dynamic environment.3)Proposed a combined algorithm of depth filtering and nonlinear optimization,as well as Bayesian-based semantic fusion algorithm to estimate depth and semantic label of point cloud based on semantic segmentation output of convolutional neural networks and accurate localization result to build accurate semi-dense semantic map,which is more benefit for humancomputer interaction.4)Performing system experiment by public dataset to verifying semantic segmentation accuracy and real-time performance of the convolutional neural network,and the absolute localization accuracy of the system compared with the traditional stereo visual SLAM system and the effectiveness of the semi-dense semantic mapping algorithm.