Three-Dimensional Object-Level Semantic Mapping For Home Environment Based On Point Cloud Segmentation

With the development of science technology and continuous improvements of intelligent robot technologies,home service robots have gradually entered our people’s life.As the basic technique of home service robots,the metric map,constructed by SLAM(Simultaneous Localization and Mapping)technology,can be used to execute the navigation tasks.However,in the execution of high-level semantic information-oriented home service tasks,service robots cannot understand semantic messages and fail to perform those assignments with metric map.Therefore,on the basis of traditional SLAM technology,it has become an important issue to integrate the object detection and instance segmentation technology based on deep learning to construct the object-level semantic map which can not only express the environment messages but also transfer object semantics.The main tasks of this article are as follows:In order to solve the problem that the dynamic objects in home environment affect the accuracy of 3D map,the construction of 3D dense point cloud map based on dynamic scene processing is studied.For the detection and elimination of dynamic environment objects in home scene,a dynamic scene processing method based on neural network and traditional dynamic detection are presented to eliminate the dynamics.The lightweight object detection network and a dynamic goods dataset based on yolov5 model are used to detect the common dynamic objects in home environment using epipolar constraints,while the pyramid LK optical flow method is used to detect the environmental objects which are not in the above dataset,and the detected dynamic feature points are eliminated so as to improve the accuracy of threedimensional map.Besides,since sparse point cloud map is not suitable for 3D scene reconstruction,a dense point cloud map construction thread based on ORB_SLAM3 system with dynamic scene processing module is proposed to realize three-dimensional environment map reconstruction.In order to acquire the three-dimensional semantic information of instance objects,an object semantic extraction strategy based on projection of 3D bounding box is studied.For object point cloud segmentation,this paper proposes to adopt RANSAC algorithm to detect the plane of dense point cloud and segment the object point cloud,and introduce an adaptive threshold strategy for the influence of threshold.To realize the 3D semantic information of each point cloud clusters,it is proposed to adopt DBScan clustering algorithm based on density to segment the object point cloud into multiple sub point cloud clusters with density connectivity,and use KD tree to accelerate the neighborhood searching process for the slow seek of adjacent points in the original algorithm,so as to obtain the 3D semantic information of each point cloud cluster quickly.To realize the target of instance object semantic information,this article proposes to compare the similarity between the 2D bounding box obtained by Mask R_CNN network and that projected by 3D bounding box obtaining from point cloud cluster,which matches the sub point cloud cluster with object category.In order to build an object-level semantic map conducive to the execution of service tasks,it integrates dense point cloud map and three-dimensional semantic information to study the object-level semantic mapping method in this article.For Storing objects semantic messages,this paper proposes an instance semantic database to save and update them.The instance objects in the database are distinguished by different colors according to their categories,and the semantic information association of static dense point cloud map is realized by integrating the instance object information into dense point cloud.Aiming at the problems of large space occupied by point cloud,an obj ect-level semantic octomap containing object three-dimensional bounding box is constructed.Finally,this passage verifies the object-level semantic mapping process by experiments on public TUM datasets and simulated family environment in the laboratory.Experiments show that the dynamic environment object-level semantic mapping method proposed in this article can detect and obtain the three-dimensional information of environment instance objects,and improve the accuracy of object-level semantic map construction.