Research On 3D Semantic Mapping Technology Of Mobile Robot Based On Multimodal Information Fusion

Simultaneous Localization and Mapping(SLAM)is a core technology for mobile robot sensing and planning,and it is a hot research topic to fuse multiple sensor information from robots to achieve high accuracy localization and mapping in complex scenes.This paper takes the mainstream SLAM algorithm as the framework to study the multimodal information pose estimation and 3D semantic mapping algorithm fusing vision,inertial measurement unit and point cloud,aiming to improve the mapping and localization accuracy of mobile robots in challenging scenarios such as variable illumination,high dynamics and fast motion,and the main innovations and contributions are as follows:(1)To address the problem of low stability of feature association with traditional feature extraction algorithms in scenes such as low illumination and variable viewing angles,the feature extraction and description algorithm of improved Super Point network is proposed to improve the accuracy and reliability of feature association.In addition,to reduce the influence of dynamic objects on feature association,a dynamic feature rejection strategy integrating semantic segmentation and motion consistency detection algorithms is proposed,which can effectively associate static feature point information and construct a least squares optimization model to estimate the state in real time by multi-constraint information.The experimental results in the public dataset Eu Roc show that the proposed system improves the positional accuracy by up to 15%compared with the internationally recognized VINS-Mono system;in the dynamic dataset ADVIO,the mean value of pose error of this algorithm is reduced by 18%.(2)The visual-inertial system still suffers from scale and pose drift problems in scenes such as low texture and structural degradation.This paper proposes to fuse the point cloud information to estimate the depth of visual features to compensate for the problem of visual scale drift.The lidar odometry front-end is constructed using the point cloud,and the robot motion estimation is achieved by frame-to-map feature point cloud matching,and the acquired lidar pose constraint information is added to the visual-inertial optimization framework to jointly optimize the robot state,which can ensure high accuracy pose estimation in degraded scenes.The experimental results show that the fused lidar point cloud constraint information can effectively improve the state estimation accuracy of the system in complex scenes.(3)The pose estimation system fused with multimodal information will have accumulated errors when running for a long time.A pose graph optimization method based on point cloud loop closing correction algorithm is proposed.By re-encoding the point cloud,the loop retrieval time can be reduced,and the accumulation can be effectively eliminated for loop closing scenarios error.In order to realize the culling of dynamic point clouds and create a global static 3D semantic map,a dynamic point cloud culling strategy based on a priori semantic segmentation network is proposed,which uses prior point cloud segmentation labels and point cloud clustering to ensure that dynamic point cloud objects are detected and eliminated.A globally consistent 3D semantic map is constructed by combining global pose and segmentation point cloud increment.