Research On Real Time 6DOF Robot Localization Based On Visual And Inertial Fusion

In this thesis,UEYE camera from IDS Company of Germany is used as the sensor of environment perception to set up the parallel binocular stereo vision system.Using the low-cost IMU(Inertial Measurement Unit)which has short-term stationarity in the image sampling interval,we aim to address the robot location problems by fusing the visual odometry with inertial navigation,so that the 6DOF(Degree of Freedom)mobile robot localization is realized with high accuracy and real-time performance.The main work is as follows:First of all,the real-time positioning problem of the robot with the inertial sensor is discussed.The common coordinate systems involved in the inertial navigation are described.Then,the common methods of characterizing the inertial navigation attitude are introduced,including the cosine matrix,Euler angle,quaternion method.The quaternion method is described in detail,including the definition and properties of quaternion,rotational quaternion differential equation and description of rotation with quaternion.Especially,the basic principles and the basic equations of the inertial navigation system are given by taking the SINS as an example.The localization process of INS and calculation of the inertial navigation attitude are described in a straightforward way.Secondly,the real-time positioning of robot with visual sensor is discussed and researched further.Experiments for visual modules are carried out in the ROS(Robot Operating System)operating system,so we first introduce the ROS operating system and its basic principles of writing and running programs in ROS.Then,the camera imaging model,camera calibration,binocular stereo image correction,feature point extraction and matching algorithm,and motion estimation are developed,respectively.The binocular camera is calibrated using the UEYE camera calibration package in ROS operating system.Finally,based on the theory of visual navigation and LIBVIS02(Library of Visual Odometry),the visual navigation code is written in ROS?and the experimental results of the video sequences are displayed by the visualization tool RVIZ and the motion trajectories are displayed in the form of point cloud.Finally,a novel method by fusing the visual odometry with inertial navigation is proposed and validated.Using the low-cost IMU which has short-term stationarity in the image sampling interval,we aim to address the robot location problems by fusing the visual odometry with inertial navigation.For the visual odometer motion estimation,the objective function for solving the motion parameters is established by minimizing the re-projection error.To fuse visual odometry with inertial navigation,the rotation matrix and translation vector calculated by inertial solution are used to reconstruct the objective function.The final rotation matrix and translation vector are solved by Gauss-Newton method.Based on the above theories and platforms,the code for the fusion scheme in ROS is developed,where the experimental results using RVIZ are demonstrated.