Research On Real Time Navigation And Localization Model And Method Based On RGB-D Camera

RGB-D camera,consisting of a RGB camera and a depth camera,is a new type of sensor that can directly and simultaneously obtain both visual texture information and per-pixel depth information.Therefore,RGB-D camera has a natural advantage for visual-based localization and mapping.In this research,the theory and method of autonomous localization of moving platform based on RGB-D camera in GNSS-denied environments are studied.A rigorous geometric model of RGB-D camera is established,a high precision autonomous localization model is proposed,and real time localization and mapping method and software system are developed.The main research contents and are described below.1.The rigorous imaging geometric model of RGB-D camera is studied and established.The strict projection relationship among 3D space,depth data and visual texture data is established based on the imaging principle of RGB-D camera.The traditional collinearity equation has been extended by incorporating the geometric model of depth measurement.The imaging characteristics of the RGB camera and the depth camera,the relative exterior orientation of these two cameras,the measurement precision of the depth camera are obtained by camera calibration.Finally,the rigorous geometric of RGB-D camera which contains both the depth information and texture information is established.2.A new method of RGB-D camera localization is proposed based on extended bundle adjustment with integrated 2D and 3D information.The traditional bundle adjustment,which refines the camera pose only using visual information,is extended by incorporating the depth information into the bundle adjustment model on the basis of the rigorous imaging geometric model.The detection and matching accuracies of SIFT and ORB features are verified by designated experiments.Based on the accuracies of visual features and depth measurement,the covariance matrices of the two kinds of observations are constructed so that to complete the mathematical model of RGB-D camera.Experiments have been carried out in variety of GNSS-denied environment,such as underground tunnel,indoor environment and outdoor deserted field.The effectiveness and accuracy of the proposed method are analyzed through these experiments.3.A real-time localization and high accurate 3D mapping system is developed.The ROS platform is used to realize the real-time data acquisition with Kinect version 1 and Kinect version 2.Real-time localization and attitude determination is realized on the basis of ORB-SLAM system.At the same time,high precision 3D mapping is achieved by optimizing the output of the real-time system through large scene 3D data tracking and optimization using ElasticFusion system.