Semantic Parsing Of Dynamic Scene Based On RGB-D Video Sequences

Nowadays visual SLAM has been widely applied in various robotic scenarios,at the same time,high-level visual tasks require a scene model with increasing semantics.Due to the effectiveness of deep learning methods in computer vision field,many studies replace the functional modules with neural network layers in some specific scenarios.However existing methods have some deficiencies in application.On the one hand,classic SLAM method made static assumption of the scene,the occlusion and motion interference of dynamic objects will reduce the tracking accuracy and modeling performance.Geometric method or deep learning network method can eliminate the dynamic objects in the scene,they abandoned the dynamic and semantic information,in this way dynamic information in the scene are not used effectively to optimize the SLAM system.On the other hand,semantic models match semantic annotations with map points rather than mark the instance-level semantic annotations.And the lack of dynamic objects in modeling process resulting in an incomplete semantic description of the scene.In order to solve the above problems,the DSP-SLAM(Dynamic Semantic-Parsing SLAM)provided by this thesis does the following work:Firstly,based on the existing motion detection and deletion methods used in the SLAM front-end process,DSP-SLAM redefined the standards of culling the dynamic objects,and improves the ICP algorithm.The semantic ICP algorithm in this paper make a decision whether dynamic objects participate in the calculation of camera poses to improve the tracking accuracy of the system in dynamic scenes without performance loss.Secondly,DSP-SLAM proposed a back-end factor graph optimization algorithm that combines self-constraint motion and semantics construction,the modeling of factor graph and the derivation process of objective optimization function are also given.Thirdly,DSP-SLAM constructed octrees model for both static and dynamic objects,and maintained both static and dynamic semantic results in the scene model.So that the dynamic objects can be displayed correctly,and the position of dynamic objects were updated in time.Experiments show that,the ATE accuracy of DSP-SLAM on TUM dynamic sequence reached the centimeter level,which improved one order of magnitude compared with ORB SLAM2.By using the semantic ICP algorithm,the front-end of DSP-SLAM reached higher accuracy than Dyna SLAM in TUM static sequence.The back-end of DSP-SLAM improved factor graph optimization based on GTSAM by dynamic and semantic factor fusion.Compared with the basic pose graph optimization in ORB SLAM2,the ATE of DSP-SLAM reduced by 20%,which proved the scientificity and effectiveness of DSP-SLAM.Finally,a semantic scene model with dynamic annotations was given by DSP-SLAM.