Indoor Three-Dimensional Hierarchical Semantic Mapping From RGB-D SLAM

With the development of Simultaneous localization and mapping(SLAM)and the development of 3D scene parsing based on RGB-D images,semantic SLAM is getting more and more attention.Although there have already been many researches that combine 3D realtime reconstruction and semantic segmentation to get a global 3D map with object-level semantics,these studies did not consider the granularity and organization of semantic elements of the map.For a robot to perform a general task,the execution process is not a single operation but consists of multiple stages.Different stages of the task require different granularity of semantics.The mismatch of semantic information granularity will increase the computational burden of the robot performing the task,thereby making the semantic map less useful.Based on the fact that the main geometrical structure of the indoor scene is planar,this paper proposed an online semantic mapping method that includes geometric-planar,layout and object information which are organized in a hierarchical fashion.Based on the relationship between different hierarchies of semantic information,we obtain multihierarchical semantic information via a fast scene parsing method that uses the results of the planar structure parsing within a frame and the structured geometric information from the global model.We proposed a new planar region representation method to structure and denoise planar regions,which makes it possible to compress the amount of model data while adding more semantic information.An adaptive data fusion method that for a plane gridsurfel hybrid model representation is designed to do the reconstruction of the scene.We then do model optimizations by taking the advantages of the global semantic map.Model optimizations include: Reduce the impact of sensor noise and the instability of plane detection algorithm base on semantic relations,merge the split planar faces by semantic information,do scene completion by using the semantic information.Under the stable recognition of the layout plane in the scene,a floor plan map of the interior can be automatically generated.Finally,we can obtain a global hierarchical semantic map with geometric,floor plan and object information.Such a hierarchical semantic map can cope with the need of different semantic granularity of users in different scenarios.Experiments show that the intra-frame planar structure parsing optimized by semantic information has a greater improvement in the recall rate than the comparison method.The hierarchical semantic map achieves a higher data compression rate than the SLAM algorithm's point cloud map.