Study On Visual Semantic SLAM

In an unknown environment,using the camera to autonomously determine its own pose and build a map of the space without prior information,which is what SLAM(Simultaneous Localization And Mapping)needs to do.After years of development,the traditional visual SLAM framework has achieved many results and has been widely used in many scenarios.Among them,trying to locate itself and building a map through sparse feature points is one of the current mainstream methods.However,the robustness and accuracy of traditional visual SLAM have been greatly challenged under the imaging conditions such as complex weak texture scenes and severe lighting changes.In recent years,with the development of deep learning technology,the use of deep learning methods to solve problems in visual SLAM has become a research hotspot,but the current end-to-end SLAM model is difficult to achieve efficiency and accuracy comparable to traditional methods.However,deep learning is good at extracting the deep features and semantic information of images.By optimizing the network model design and combining it with traditional methods,it is expected that a more robust SLAM method can be obtained.This paper uses deep network to extract feature points and semantic category information,and integrates them into traditional visual SLAM to realize a semantic SLAM method.The main research contents and contributions are as follows:Firstly,aiming at traditional feature points that are completely dependent on the geomet-ric information of the image,they are easily affected by problems such as illumination,noise and image blur,this article uses deep learning methods to extract feature points and descrip-tors.Use virtual datasets to solve the problem of lack of labeled datasets,and migrate to real scenes through data enhancement.In terms of engineering,we trained the applicable dictionary for the bag-of-words model,and at the same time,we learned from the process in the tradi-tional framework.We meshed the feature map output by the network and used high and low thresholds for dynamic extraction to ensure that there are sufficient number of feature points and distribute as evenly as possible.Finally,the feature points are integrated into the traditional SLAM framework and verified by experiments.Secondly,in order to mine and use higher-level information of images,this paper proposes a SLAM system based on semantic category information,which uses an optimized real-time semantic segmentation network to obtain semantic information of images and make full use of semantic information.First of all,we perform morphological expansion on the segmentation information of dynamic object categories to make up for the problem of semantic segmenta-tion accuracy and filter out the corresponding feature points.The semantic score is calculated according to the segmentation information of the image,and the frame with the highest score in the candidate keyframe queue is taken out as the keyframe to increase the amount of infor-mation of the keyframe.By continuously updating the semantic statistical information of map points,a semantic map is established,and semantic weights are added to matching map points when optimizing poses and map points to improve the accuracy of optimization.Finally,the effectiveness of the method is verified through experiments.