Indoor Visual SLAM Based On Object Semantic Information

Visual simultaneous localization and mapping(SLAM)refers that robots use visual sensors to complete its localization and map construction.In recent years,with the wide application of robots in the fields of social services,public safety and disaster relief,traditional geometric-based visual SLAM cannot meet the needs of high-level tasks of robots,and robots need to perform interaction and collaboration tasks from the semantic level,so how to improve the semantic perception and understanding ability of robots is the key to improve the level of robot intelligence.As an abstracted knowledge,object semantics can not only improve the robot's ability to understand the surrounding environment,but also improve the ability of the robot to cope with complex environments.For indoor scenes,this paper systematically studies how to use object semantic information for visual SLAM,including the following three aspects:To handle the problem that dynamic objects interfere with visual odometry,this paper proposes a visual odometry algorithm based on dynamic objects removal.By integrating dynamic object detection module into ORB-SLAM2 visual SLAM system,the improved algorithm reduces the impact of dynamic objects on visual odometry and improves the ability of ORB-SLAM2 to cope with dynamic environment.For effective detection of dynamic objects,this paper designs a dynamic object detection method based on object category and optical flow method.This method is based on YOLOv3 object detection network to acquire potential dynamic objects,and combined with optical flow method to complete the detection of dynamic objects.The experimental results show that compared with other visual SLAM,the improved ORB-SLAM2 has better pose estimation accuracy in dynamic scenes.To handle the problem that visual feature based camera relocation algorithm is sensitive to complex environment such as illumination and occlusion,this paper proposes a camera relocation method based on object properties.The method firstly completes the construction of visual semantic database based on static object detection and recognition in key frames.In order to obtain the static objects in the key frame,this paper obtains the object information in the key frame using Mask R-CNN instance segmentation network,and employs the improved visual odometry to remove dynamic objects.In the phase of camera relocation,a two-layer filtering mechanism based on object categories and image features is designed to obtain similar key frame in visual semantic database,and the pose estimation between the current frame and similar key frame is completed by combining the object features in current frame and similar key frame with bundle adjustment.The experimental results show that the camera relocation method based on object properties has higher matching efficiency and positioning accuracy.To handle the problem of multi-view object information updating in three-dimensional semantic map,this paper proposes an object-oriented three-dimensional semantic map construction algorithm.Firstly,the algorithm completes the construction of online object database based on the relocation visual semantic database.In the aspect of object database updating,this paper proposes a two-layer filtering mechanism based on object categories and object point cloud coincidence degree.Then,this paper generates a three-dimensional map using the precise pose of robots estimated by the improved visual odometry.Finally,the object-oriented three-dimensional semantic map is constructed by mapping the object information in the object database to the three-dimensional map.The experimental results show that the proposed object database updating strategy can recognize and distinguish most objects in the environment.At the same time,the constructed object-oriented three-dimensional semantic map provides the semantic data foundation for high-level tasks such as robot task planning and human-computer interaction.By introducing object semantic information in the field of visual odometry and relocation,this paper improves the ability of visual SLAM system to cope with complex dynamic environment.At the same time,the object-oriented three-dimensional semantic map construction algorithm proposed in this paper has a positive effect to improve the robot's complex scene perception and understanding ability.This research has important reference value for high-level tasks such as human-computer interaction,robot task planning and so on.