Build Semantic Scene Map Based On The Monocular Camera

For a bionic robot,the premise that it can move autonomously is that it knows what its surroundings are and where it is in the environment.This requires us to be able to process the environmental information collected by the robot sensor in real time and build scene map for it.Here we use a monocular camera as the sensor to collect information about the surrounding environment.Monocular camera is the most common camera in our daily life.It only uses a camera to shoot scenes and uses two-dimensional images to reflect the three-dimensional world.We chose monocular camera as the hardware to build the map,on the one hand,because it is cheap and widely used,on the other hand,because its depth measurement range is not limited by the binocular baseline distance like the binocular camera,and also unlike depth cameras,which can only be used indoors and cannot measure the depth at each point.Monocular camera is the only hardware that is not limited by the use scene and depth range,its only defect is that it loses the information of scene depth during shooting,which can be recovered by deep learning.In addition,when shooting scenes with a monocular camera,it will be affected by ambient light.Therefore,we will transfer the style of images captured by a monocular camera in advance,and unify the images taken at different moments into the same light condition,so as to eliminate the influence of ambient light changes on depth estimation and subsequent adjacent frame registration.Therefore,the research content of this paper is as follows: firstly,use the style transfer network to unify the brightness of the image taken by the monocular camera,and on this basis,recover the scene depth of the image with the method of deep learning;At the same time,semantic instance segmentation is carried out on the image to add semantics to the constructed map,so as to meet the needs of more application scenarios;In addition,the error of depth estimation is corrected by using the result of semantic segmentation,considering the relationship of depth at different positions on an object surface;Finally,the monocular image with scene depth in the current frame is registered with the projected image of scene map in the previous frame to obtain the pose of the robot in the current frame.With the pose,different semantic instances of the current frame image can be segmented into different instance parts of the scene map,so as to form a scene map with semantics,that is,each point in the map knows which object in the real world it corresponds to.