A Study About Guiding A Robot By The Fusion Of 2D Image And 3D Point Cloud Features

Adaptive guidance is one of the challenging issues in the realization of intelligent UAV robot. The conventional framework of adaptive guidance is based mainly on 2D image features, depending on the texture information of the image and limited by the perspective of the image. With the development of 3D sensors, adaptive guidance based on 3D point cloud data has also been a significant development. The existed methods, based on the 2D image and the 3D point cloud to track a target and guide the UAV robot, are mainly used the block features of 2D image and the depth information of 3D point cloud, but it do not well mix the features of the two images.This paper aims to study the visual tracking algorithm based on multi-sensor data fusion. And presents a approach of tracking a target and guiding UAV robot by merging 2D image and 3D point cloud features. First, integrates the 2D image FAST corner feature and color feature to track the target and guide the UAV robot. Second,the use of 2D image SURF feature, verifies the effect guidance of the UAV robot.Final, the integration of 2D image and 3D point cloud features, to achieve steady UAV robot guidance.The main work and contributions of this paper are the following aspects:1.Analysis the 2D image features, FAST corner feature and color feature, and presents a method of merging FAST corner feature and color feature.2.The use of 2D image SURF feature, and establishes a system to verify the effectiveness of the adaptive guidance of UAV robot by SURF feature.3.The combination of 3D point cloud feature, integrated the 2D image and the3 D point cloud features to achieve the adaptive guidance of UAV robot. And with the guidance results are compared before fusion, to verify its effectiveness.