Simultaneous Localization And Mapping Based On Seamless Fusion Of Heterogeneous Multi-Modal Features

Simultaneous localization and mapping(SLAM)consists of the estimation of the robot pose as well as the map of the environment by processing the measurement data captured by on-board sensors.Nowadays,with an increasing demand for the application in domestic services,autonomous driving and so on,the SLAM techniques have gained rapid development.It is of great importance in a SLAM system to estimate the poses of the sensor and build a model of the scene using the features extracted from the environment.Different types of features have their own strengths and weaknesses in various scenes.Specifically,there exist multimodal features such as points,lines and planes in the structured environments.Hence,to increase the effectiveness of the robot system in unknown scenes,the fusion of multiple types of features has become a hot topic in SLAM field.This thesis focuses on the simultaneous localization and map building based on the seamless fusion of heterogeneous multi-modal features.First,a multi-modal feature association algorithm is proposed based on a multi-hypothesis framework.Then,the constraints provided by different types of features on the robot motion estimation are quantitatively analyzed.Based on the analysis results,a seamless fusion method is proposed to calculate the poses of the robot fusing heterogeneous multi-modal features.In this thesis,the major contributions are summarized in the following.(1)The problem of multi-modal feature association is addressed.A multihypothesis framework-based feature matching algorithm is proposed to simultaneously associate different types of features(planes and lines)and further estimate the pose of the sensor.The multi-hypothesis framework is achieved by constructing an interpretation tree(IT)structure.Specifically,an inter-node consistency is proposed for generation of hypotheses and a consistent transformation model(CTM)for each hypothesis is explicitly expressed and incrementally updated.When the IT is constructed,a closed-form solution to the feature association and the pose estimation can be obtained.Then,a multi-modal feature joint optimization method is introduced to further refine the pose estimate and parameters of features.During the optimization,different types of geometric features are appropriately parameterized and the uncertainties arising from feature extraction are derived and used to balance the contributions of multiple types of features in the cost function.Extensive experiments are executed on public datasets and the results demonstrate that the proposed method can achieve higher accuracy and stronger robustness.(2)A plane-line-based RGB-D visual odometry(PLVO)is proposed to address the correspondences between the degenerate cases in the pose estimation and the spatial configurations of features.First,the plane-line hybrid association graph(PLHAG)is proposed to describe the spatial configurations of different features as well as the geometric relationships between them.Then,the pose of the sensor is estimated based on the adaptive fusion of planes and lines.Specifically,an adaptive weighting algorithm is proposed based on correspondences between the degenerate cases and the spatial configurations of features,considering the geometric relationships between plane and line features.For the degrees of freedom(Do Fs)of the pose that cannot be constrained by planes,the line features are supplementarily used to obtain the full 6Do F pose estimation of the sensor and solve the degenerate problem.Various experiments on public benchmarks as well as in real-world environments demonstrate the efficiency of the proposed method.(3)A SLAM system is achieved based on a seamless fusion of heterogeneous multi-modal features(plane-line-point).First,a probabilistic fitting algorithm for the geometric feature is proposed to compute the parameters of planes and lines.By exploiting the error model of the depth sensor,the proposed probabilistic fitting is adaptive to various measurement noises corresponding to different depth measurements.As a result,the estimated parameters are more accurate and robust to the points with large uncertainties.Then,a constraint analysis is performed to quantitatively measure the constraints provided by different types of features on the pose estimation of a sensor.Using the results of the constraint analysis,a plane-line-point fusion-based SLAM method is proposed.Through the fusion of multi-modal features,both the structure and texture information is fully exploited and the problem of pose estimation remains well-posed in all circumstances.In addition,The comparison results of extensive experiments on public datasets demonstrate that the SLAM system seamlessly fusing multi-modal features can achieve high accuracy and robustness.