Dense Semantic Mapping Based On Deep Learning

Slam(simultaneous localization and mapping)is the key technology for robot to achieve intelligent and autonomous decision-making.The traditional visual slam is mostly based on the assumption of static environment,and the map only contains the geometric information of the environment,doesn’t contain any semantic information of object level.The perception algorithm based on deep learning can recognize different objects in the environment.The combination of slam and deep learning is the current research hotspot.In this paper,we use instance segmentation to obtain the semantic information of the objects in the environment,and combine with optical flow to detect moving objects in the environment.We only reconstruct the static objects and the background.The dense semantic map can enhance the environmental perception ability of the robot,and achieve the robust localization and mapping in the indoor dynamic scene.For instance segmentation,the traditional real-time optimization methods based on geometric segmentation mostly rely on the accuracy of depth information,which has the problems of low robustness and accuracy in complex scenes.In this paper,a real-time optimization method based on dense optical flow is proposed.A certain number of frames are segmented in each interval,and the motion mask of frames without instance segmentation is predicted by dense optical flow.Experimental results show that the algorithm can meet the needs of dense semantic mapping and moving objects removal.In view of the low accuracy of traditional SLAM Algorithm in dynamic scene,this paper proposes a semantic dynamic SLAM algorithm based on instance segmentation and optical flow.The frame buffer queue is used to synchronize segmentation thread and tracking thread.Based on the sparse optical flow and the distance from the matching feature points to the polar line,the motion consistency is detected.Combined with the example segmentation,according to the proportion of the moving feature points in the mask,we can judge whether the object moves or not,and get the motion mask.In the feature extraction process of the odometry,only extract the feature points in the static area,and the feature points are managed by quadtree to ensure the uniform distribution of the feature points in the static area.In the whole SLAM pipeline,only static feature points are added to the global map,and only static feature points are used in closed-loop detection,which improves the localization accuracy and robustness.The experimental results on the indoor open dataset show that the localization accuracy of the algorithm in most scene sequences is greatly improved.The traditional slam map does not contain semantic information,and its accuracy is low in dynamic scene.This paper builds dense semantic map based on semantic dynamic slam module and truncated symbol distance model.By building the object database,the instance level data association between frames is achieved,and the instance mask of each frame is obtained.According to bounding box of the moving object to remove moving object in the reconstruction process.Then,we use Bayesian model to update the foreground probability and instance label of voxels.The experimental results in the open dataset show that the method can recover the spatial structure and surface texture of the environment better in both static and dynamic scenes.The map constructed in this paper contains the instance level semantic information of objects,which can meet the needs of subsequent robots to perform intelligent interaction,semantic navigation and other advanced tasks.Finally,the research work of this paper is summarized,and the future research work is prospected.