Research On Semantic SLAM Method For Fusion Of Solid Of Revolution

Intelligent robots need to understand the geometry and semantic information of the surrounding environment in order to interact meaningfully with the scene.The special structure and semantic information in images have proven to be a promising direction for visual SLAM(Simultaneous Localization and Mapping,SLAM).In this paper,a visual SALM method fusing the structure and semantics of SORs(solid of revolutions)is proposed.The main innovations include two aspects: 1).The automatic recognition and high-precision segmentation method of revolving bodies combining the deep learning and the traditional image segmentation algorithm;2).The general framework of the semantic SLAM that fuses the revolving structure,and the scale and pose fusion method of the revolving structure in the SLAM point cloud map.Based on the traditional visual SLAM,this topic combines the structure and semantic information of revolving bodies.Firstly,by training the Mask R-CNN(Regional Convolutional Neural Network,R-CNN)revolving body recognition and segmentation network,combined with the traditional image segmentation method,the outer contour of the revolving body is extracted;Secondly,according to the special geometric constraints in the SOR objects imaging,the scale model is established according to single image;Finally,the SLAM system framework with revolving structure is designed,and the scale and pose fusion method of the revolving body is proposed.The semantic information of the revolving body in the scene is used to help the SLAM back end to perform closed-loop detection and establish a structural semantic SLAM map.The specific work of this paper can be summarized as follows:(1)SORs detection and segmentation.In order to make computers understand the revolving structure in the image,this topic collects the photos of SOR from Internet,including vases,porcelain,water cups,etc,and uses image annotation tools to manually mark and create the image dataset of SOR.Using this dataset,the Mask R-CNN object recognition and segmentation network is trained.The results of the Mask R-CNN SOR segmentation are used as the priors of the Grabcut active image segmentation algorithm to obtain the initial segmentation results of SORs;then Canny edge detection is performed in the region of the SOR.Finally,the Mask R-CNN segmentation results,the Grabcut segmentation results and the Canny edge detection results are merged to realize the automatic segmentation of the revolving body,and the classification information of the revolving body and the outer contour pixel coordinates are obtained.(2)Metric 3D Reconstruction of SOR from single image.In the process of imaging of revolving structure,there are special geometric constraints.This topic uses the outer contour of SORs in a single photo to reconstruct a scale model of the revolving bodies.Firstly,the elliptical equation of the section of the revolving body is fitted according to the outer contour of the revolving bodies.Secondly,according to the elliptical spatial relationship constraints of the cross section of the revolving body,and the analogy of the revolving structure and the single axis motion,the constraint equations are established,and the image of absolute conic curve is solved to obtain the camera internal parameter.form a side profile and a section ellipse imaged by the revolving body,using the planar homology constraints existing in the image of the revolving body to solve the image of meridian;then,using the vanishing line of the elliptical plane of the section,the imaging symmetry axis of the revolving body,and the camera internal parameter,Plane correction is performed on the calculated meridian and the symmetry axis of the revolving body.Finally,the meridian is normalized and rotated around the axis of symmetry to obtain a scale model of the revolving body.(3)Research on visual SLAM method for fused structure.Based on the scale modeling method of revolving body,the overall framework of the visual SLAM system with SORs is designed.The method of the scale and pose fusion of the revolving body is proposed.The semantic information of SORs is helpful to the closed-loop detection of SLAM,and the structural semantics SLAM map is established.Based on the assumption that the top or bottom ellipse center points of the same revolving body are matching points between two different frames,the true scale and pose initial estimation method of the revolving body is proposed.By using the sample points on the image of meridian,a least squares optimization problem is constructed,and the scale and pose of the SORs are accurately solved.This subject combine deep learning and the traditional methods to obtain the structure and semantic category information of the SLAM video sequence,and realizes the enhancement of the traditional SLAM point cloud map.Experiments and analysis show that the semantic SLAM method of fusing revolving structure proposed in this paper can be used in the actual SLAM scene to establish a SLAM map containing the semantics and complete structure information of the revolving body.Compared with the point cloud map,it can be used for higher Hierarchical application scenarios.Unlike previous efforts in this domain,the shape of SOR is calculated directly,without the need of SLAM point cloud map information.In addition,the proposed method can even recover the structural information of transparent SOR objects.