Research On Panoramic Image Stitching And Panoramic Visual SLAM

Image stitching and simultaneously localization and mapping(SLAM)are traditional research directions in the field of computer vision,but there are still many problems in practical applications.For example,in image stitching,the recent research work mainly focuses on how to solve the misalignment problem when there is a large parallax between cameras.As for panoramic image stitching which is aiming at acquiring a full-view image,under the premise of solving the misalignment,how to ensure the sphere-consistency of panoramic image has always been a neglected problem.In addition,in existing vision-based SLAM researches,the SLAM system generally cannot work normally under the pure rotation motion of the camera or when the scene texture information is poor.Recently,some researches show that the larger field of view is beneficial to alleviate these problems.Therefore,our works in this thesis include the following two parts:(1)we propose a panoramic image mosaic algorithm based on spherical local alignment estimation.The spatially-varying warp estimation is a method of image stitching proposed in recent years,which can effectively solve the problem of stitching misalignment caused by camera parallax.However,in the panoramic image mosaic,the stitching result we actually require is corresponding to a closed spherical surface.If the local alignment estimation is performed only on the plane,the uniformity of the closed spherical surface cannot be satisfied.Therefore,in the mosaic of panoramic images,we propose to perform local alignment estimation directly on the spherical surface.At the same time,in order to obtain seamless stitching results around low-texture regions,we combine the alignment information of template matching among low-texture region with the original feature-based local alignment estimation to obtain seamless panorama results.(2)Based on the existing ORB-SLAM framework,we propose an ORB-OmniSLAM which takes the panoramic image holding the largest field of view as the input.In ORB-OmniSLAM,in order to keep the invariance of existing image features algorithm,we propose to use a cube map to represent the input panoramic image,and introduce a cube vector map to solve the discontinuity of the cube image space.In view of the cube vector map,we modify the following parts of the original ORB-SLAM:(a)propose a map initialization algorithm based on the view direction vector,in which we calculate the camera essence matrix from matched view direction vector between the two frames,then restore the motion parameters and triangulate the initial map points.(b)proposes a pose estimation algorithm based on the view direction vector,using the matched pairs of map point and direction vector to estimate the camera pose directly;(c)proposes a triangulation algorithm based on the direction vector to linearly estimate the three-dimensional coordinates of the map points with the known camera pose;(d)apply the approximate spherical distance as the distance loss term of optimizations instead of the traditional distance loss defined on image plane.