Semantic Mapping Of Indoor Environment Based On Deep Convolutional Neural Network

Simultaneous localization and mapping(SLAM)is a key step for mobile robot to perform tasks autonomously.The traditional methods can only distinguish obstacles and passable areas,and don't contain the information about object category.However,the perception algorithm based on deep learning can understand the image.The combination of these two aspects makes hotspots of recent research.Taking advantage of deep learning in scene recognition and SLAM in autonomous location,we propose an algorithm of environment semantic mapping based on deep learning,which has strong theoretical and practical significance.As for the input image sequence,we use ORB-SLAM to select keyframes and estimate the pose between keyframes.In order to reduce redundancy and excessive time consumption,the image segmentation is performed only on keyframes.During the pose estimation,multiple models are used to obtain original pose and then make local and global optimization on it,so as to improve the accuracy and stability of localization.The semantic segmentation in this paper is based on Deeplab,aiming at overcoming its two shortcomings:Firstly,in view of the un-learnable characteristics of upper sampling or reverse pooling,the convolution upper sampling network is proposed to map features to higher dimensions and transform them to appropriate dimensions,thus completing the transformation of feature sizes.As for the small objects failing to rebuild since the objects vary greatly in size,we put forward a kind of depth-gating.If we can't get the depth directly,the deep convolutional neural network can be used to generate the depth image.When we have the depth image,it can be regarded as the gating signal to control the size of dilation ratio,which enables the distant objects to keep small details and bigger objects to keep field of vision.The specific method of depth gating is that we discretize the depth into five levels,using the smaller step ratio for distant objects and the larger one for nearby objects.In addition,the analysis also finds that although the estimated depth of deep convolutional neural network may not be accurate enough,it has no effect on the improvement of segmentation due to its smoothness.On the basis of theoretical research,this paper conducts verification on the open data sets of indoor and outdoor scenes,and experimental results show that the proposed algorithm has an overall improvement for most categories.To build three-dimensional semantic map of the environment for navigation tasks,a superpixel segmentation algorithm based on SLIC algorithm is studied,playing a role as a constraint term of high order potential energy in the process of map optimization.In order to construct the three-dimensional map,we should firstly align the color image and depth image,project pixel points back into three-dimensional space by using the spatial correspondence between adjacent keyframes,then convert the dense point cloud into OctoMap.In the process of forming 3d semantic map,image semantic segmentation information and super-pixel segmentation information are merged into dense occupying map in the form of Bayesian update.The integrated semantic map is optimized in the high-order conditional random field,making the result that not only the categories is allocated in each grid considered,but also the category constraints in the super-pixel blocks are taken into account.Finally,a three-dimensional semantic OctoMap is formed to save memory and meet the needs of subsequent robot navigation tasks.Finally,the research work of this paper is summarized and the future research direction is analysed.