The Research And Implementation Of Semi-dense Semantic Mapping For Visual SLAM

Map is an important tool for human to understand the world.Map can not only depict geographical terrain of environment vividly,but is also helpful for task planning.As mobile robots are widely utilized in the industries and daily life of human beings,it is indispensable for robots to get a grip on environments with the aid of maps.Simultaneous localization and mapping is the key for mobile robots to understand unknown environments.Robots estimate their pose and create maps of environment through their sensors during travelling.Visual SLAM(Simultaneous Localization and Mapping)has received extensive attention and has been exhaustively studied in recent years due to its low cost,rich image information,technical difficulties and etc.In this paper,a method of semi-dense semantic mapping for visual SLAM is proposed.Using the poses estimated by visual SLAM,not only the semi-dense inverse depth information of the environment can be estimated,but also the objects of interest can be recognized.The main researches of this paper include:1)Implement inverse depth estimation of map-points based on depth filters and estimate inverse depth in the pixel level.Select appropriate map-points in the key-frames and use a “rough-accurate” matching method to observe and update the inverse depth estimation of mappoints according to reference-frames.Also,a key-frame selection strategy is proposed according to the representation of inverse depth.2)Implement inverse depth map estimation of key-frames based on image pyramid and estimate inverse depth in the image level,including inverse depth map update and propagation.In the estimation process,the inverse depth map of each layer in the key-frame's image pyramid is calculated.When the estimation is finished,the inverse depth maps of the pyramid is passed from top to down.3)Implement 3D recognition of objects of interest based on Mask R-CNN(Region-based Convolutional Neural Network).The recognition result is converted to a specific form of images as environment semantic information,and a novel map representation form is used to fuse all information of key-frames including poses,images,inverse depth,error variance and object recognition.4)Design and implement a complete semi-dense semantic mapping system for visual SLAM,including a system startup module,an inverse depth map estimation module and a map module,where the “semi-dense” is implemented in the inverse depth map estimation module and the “semantic” is implemented in the map module and they are fused via the map module.The experimental results show that the semi-dense semantic mapping method for visual SLAM proposed in this paper can accurately estimate the inverse depth maps of key-frames and successfully recognize the full objects of interest in key-frames,and the map representation introduced in this paper can effectively fuse various information of key-frames.The semi-dense semantic pointcloud converted from the map representation can both accurately restore the environment and identify the objects of interest in it.