Self-Supervised Deep Feature Based Visual Simultaneous Localization And Mapping System

In the field of Robotics,Visual Simultaneous Localization and Mapping technology(VSLAM)has been widely used in indoor/outdoor robot localization,robot navigation,obstacle avoidance,robot human-computer interaction,or other fields.Feature-based VSLAM is widely used because of its high accuracy and efficiency.However,feature detection algorithms tend to demonstrate poor performance under camera shooting lighting changes,viewing angle changes,noise interference,and other factors.Due to this drawback in robustness,the position drift and mismatching of the extracted feature makes VSLAM's localization accuracy drop or even tracking loss in environments with complex shooting conditions.To solve this problem from the perspective of feature detection,this thesis uses self-supervised learning methods to design and train a deep convolutional neural network for feature detection and description.A self-supervised feature VSLAM system is built to improve the accuracy and robustness of the VSLAM system.The main work of this thesis is described as follows:First of all,this thesis aims to improve the robustness of the VSLAM system from the perspective of the feature detection algorithm.Through the analysis of different modules of VSLAM and the researches of domestic and foreign scholars,it is found that the feature detection algorithm is of great significance to the VSLAM system.Therefore,this thesis proposes to apply a highly robust deep convolutional neural network as the feature detection algorithm,thereby effectively improving the performance of VSLAM systems in complex visual environments.Second,this thesis trains a feature detection network in a self-supervised manner.Based on the deep convolutional neural network,the network consists of a shared feature extraction layer,a keypoint extraction layer,and a descriptor extraction layer.This network simultaneously locates keypoints and extracts 32-dimensional descriptors from the input image.To avoid the complicated labeling of supervised learning data,this thesis designs and implements a self-supervised learning method for feature detection networks.It uses artificial synthetic datasets,2D image transformation data enhancement,synchronous training and data mining to implement iterative network training,aiming to extract image features with high repeatability and high robustness.Finally,this thesis builds a self-supervised feature VSLAM system with high robustness and localization accuracy.The VSLAM system uses the self-supervised feature detection network as the frontend visual odometry feature detection algorithm and implements motion estimation based on frame-wise matching along with motion BA optimization.Backend non-linear optimization applied Bundle Adjustment to minimize the tracking error.The system also trains the bag-of-words model with the feature detection network for closed-loop detection and relocalization.To limit the memory occupation of the feature descriptors output by the feature detection network,this thesis applies a quantization method to greatly reduce its storage space while maintaining its accuracy.The quantized descriptors provide high matching performance and lightweight to the VSLAM feature landmarks.The VSLAM system demonstrates its high accuracy and robustness in experiments on datasets and real-world scenarios.To verify the robustness of the self-supervised feature VSLAM proposed in this thesis,self-supervised learning feature detection experiments,VSLAM dataset experiments,and VSLAM realistic scene experiments were performed.In the experiments,the feature detection network manages to detect keypoints with high repeatability and shows excellent matching performance under the premise that the descriptor memory is relatively low.The self-supervised learning feature VSLAM has demonstrated its high robustness and high accuracy in dataset experiments and real-world experiments.It proves that the application of self-supervised learning features effectively improves the robustness and localization accuracy of the feature-based VSLAM.It can make the VSLAM more stable in tracking and localization of cameras or mobile robots during operation.