Research On Camera Localization Based On Video Sequence

In recent years,robotics,autonomous driving and augmented reality have received extensive attention from academia and industry.Thanks to the ubiquity of cameras in daily life,image-based camera localization is a basic and key technology in the abovementioned fields,with great application potential in a variety of scenarios.For example,sweeping robots that are capable of planning paths,service robots that autonomously move and explore the environment,and unmanned vehicles that apply the technology of multi-sensor information fusion.Image-based camera localization refers to six-degreeof-freedom pose estimation of the camera with respect to a map based on the current image,including the position coordinates and the rotation angles around the three coordinate axes.The biggest challenge at present,for the task of image-based camera localization,is how to precisely localize the camera.The high-precision map is the integration of prior knowledge of the surrounding environment,and is also an important basis for judging the relative positioning relationship between the camera and the surrounding environment.The precise localization methods are different,depending on whether there is a high-precision map of the surrounding environment and the accuracy of the map.According to whether the map has been built,current approaches for image-based camera localization are generally divided into three categories:the scene is completely unknown,the scene is partially known,and the scene is completely known.Aiming at these three different types of scenario information,this dissertation studies the corresponding image-based localization methods.The main contributions of this dissertation are listed as follows.(1)An RGB-D camera localization method based on submap-joining is proposed.Aiming at a video sequence captured in a completely unknown scene,a RGB-D SLAM framework based on submap fusion is studied,including the selection of feature points and the automatic segmentation of the video sequence,as well as the detection of a loop closure between video segments.A submap-joining algorithm is applied to obtain the global consistence of the camera motion estimation.The proposed method is tested on a public dataset,and it performs robustly and quickly in this complex unknown environment.(2)A monocular camera localization method based on a loose coupling of SLAM and 3D object tracking is presented.Aiming at a known three-dimensional mesh model of an object in a scene,a monocular camera localization method based on a loose coupling of SLAM and object tracking is studied.Firstly,the monocular SLAM technology is applied to obtain the preliminary camera pose for each frame,and the object tracking technology is then used to fine-tune the camera pose,finally the accurate pose estimation of the camera with respect to the object is computed.The proposed approach obtains higher accuracy compared with the pure SLAM algorithm,and achieves higher robustness compared with the object tracking algorithm.(3)A visual global localization method based on image retrieval and learned features is proposed.Aiming at the historical database information of a known large scene,including an image database and the corresponding three-dimensional point cloud,a global localization method based on the image retrieval technology and learned features is studied.A learning-based feature is selected as the image descriptor to cope with the drastic visual discrepancies between a query photo and prerecorded database images.A hierarchical sitemap is constructed during the off-line stage,which is utilized to offer a convenient query/track of the image/geometry data,and helps to improve the accuracy of localization during the on-line stage.A global image descriptor based on image statistics is designed,which is used to re-rank the image retrieval results and improves the efficiency of on-line localization.A pose verification algorithm is improved to select the correct camera pose more reliably among pose candidates.The proposed approach outperforms the state-of-the-art methods in terms of visual localization validity and accuracy.