Semantic SLAM Based On Instance Segmentation In Dynamic Scenes

With the widespread application of SLAM(Simultaneous Localization and Mapping)technology in robotics,more and more robots are used in complex and dynamic indoor environments,such as hotels,hospitals,shopping malls,etc.These application scenarios pose higher requirements for robots,which not only need to accurately locate their position and build environmental maps in these dynamic scenes,but also need to understand the higher-level semantic information in the environment to make more intelligent decisions and actions.However,current visual SLAM algorithms have problems such as low positioning accuracy and poor robustness in dynamic environments.Therefore,aiming at this problem,a visual SLAM algorithm for dynamic environments was designed,which improved the positioning accuracy of the SLAM system in dynamic environments and built corresponding semantic maps under the premise of ensuring real-time performance.Firstly,the basic theory of visual SLAM is introduced,and the influence of dynamic objects on SLAM algorithms is analyzed.A semantic annotation thread is added to the original ORB-SLAM2 framework to distinguish between dynamic and static objects.In order to ensure the real-time performance of the system,this thread uses the instance segmentation network YOLACT++(You Only Look At Coefficien Ts)to extract semantic features of objects in the scene and encode object colors.Secondly,since animals can endow static objects with dynamic properties,using only an instance segmentation network cannot completely eliminate dynamic objects.Therefore,the dynamic region detection thread composed of the optical flow estimation network Lite Flow Net2 and YOLACT++ is used to replace the semantic annotation thread,which can completely segment the dynamic objects in the image and generate a dynamic region mask.The mask is dilated by morphological processing to compensate for the under-segmentation of contours when segmenting dynamic objects.Then,this mask is passed into the visual SLAM tracking thread to remove the feature points of the dynamic region in real-time,and use the remaining static feature points in the map for tracking and mapping.A semantic point cloud construction thread is added to the SLAM system,which combines the two-dimensional image after removing dynamic objects with the depth information collected by the RGB-D camera.While introducing semantic information into the map,it effectively reduces the impact of dynamic objects on the construction of three-dimensional semantic maps.Finally,based on ROS(Robot Operating System),an experimental platform was built.Experiments on pose estimation and semantic map construction were carried out in static and dynamic environments using TUM public dataset.The experiments showed that: The semantic map construction algorithm for static environments can effectively identify and match object instances in the environment.The semantic map construction algorithm for dynamic environment can remove feature points of dynamic objects in dynamic environment and perform real-time semantic mapping,which improves the positioning accuracy and robustness of SLAM mapping in dynamic environment.