Visual Simultaneous Localization And Mapping Based On Panoramic Annular Imaging

Visual Simultaneous Localization and Mapping(Visual SLAM)is defined as the process of estimating the camera pose(translation and rotation with respect to a reference frame)and simultaneously establishing a map of the surrounding environment,taking use of a sequence of captured images.As a fundamental technology that enables reliable positioning in applications such as space detection,intelligent robots,autonomous driving,and augmented reality,visual SLAM has received extensive attention from researchers at home and abroad in the past decade,and has achieved impressive results.At present,the traditional pinhole camera based on perspective projection is still the mainstream sensor for visual SLAM.However,the pinhole camera generally has a limited field of view(FOV),which will lead to insufficient robustness and accuracy when facing some challenging scenarios.Given this,visual SLAM based on panoramic annular imaging is proposed in this thesis.Panoramic annular imaging can produce a 3600 panoramic perception of the surroundings in a single shot,of which the core component is Panoramic Annular Lens(PAL).The larger FOV means that richer visual information can be captured by the camera at one time,and much more sufficient data can be used in pose estimation and map construction.This thesis firstly defines and analyzes mathematical models in the panoramic visual SLAM,including the PAL camera model,the parameterization method of panoramic visual SLAM,and the form of the Jacobian matrix.Besides,because the projection formula of PAL is significantly distinct from that of conventional pinhole cameras,the dual-view geometric relationship is re-derived under panoramic imaging,including the epipolar constraint,pose recovery,and feature point triangulation,to establish a good foundation for follow-up research.On this basis,a sparse direct visual odometry based on panoramic imaging(PALVO)is proposed.An initialization module based on the essential matrix,a "coarse-to-fine"tracking strategy and a feature correspondence search method along epipolar curve are specially designed.Experiments demonstrate that PALVO has excellent robustness to rapid motion and dynamic scenarios,approximate same accuracy as state-of-the-art(SOTA)visual odometry,and high efficiency.Then,a panoramic annular semantic visual odometry(PASVO)is presented to further eliminate the influence of moving objects.PASVO combines PALVO and semantic segmentation of panoramic annular images by deeply coupling semantic information to various modules of the visual odometry.In the pose estimation stage,semantic information weighting is utilized to reduce the interference of moving objects.In the mapping stage,semantic information is employed to guide the keypoint selection and feature correspondence search.Compared with PALVO,the robustness and accuracy of PASVO in practical applications are further improved.At the same time,the introduction of semantic segmentation allows a higher-level understanding of the environment besides pure geometric information.Finally,PALVO is extended to a complete visual SLAM system by integrating closure detection and global optimization to suppress the accumulated error and scale drift in PALVO.To ensure computational efficiency and achieve reliable loop detection,a hybrid keypoint selection strategy is proposed.Benefiting from the 360° imaging characteristics of PAL,PA-SLAM can handle loop closure in different travel directions,exhibiting obvious advantages compared with the classical visual SLAM based on forward-looking pinhole camera,which can only deal with loops in the same travel direction.The experimental results indicate that PA-SLAM significantly reduces the error accumulation and scale drift of PALVO,achieves equivalent accuracy of SOTA visual SLAM,and maintains the original robustness and high efficiency.