Intelligent Vehicle Localization Technology Research Based On Vision And Inertial Fusion

With the development of the national economy as well as technology,intelligent vehicles are in full swing as the crown jewel of mobile robots.In the unknown environment,Simultaneous Localization and Mapping(SLAM)technology based on multi-sensor fusion is the key foundation to support intelligent vehicles to achieve autonomous navigation and complete their assigned tasks.However,due to the complexity of the real driving environment,the accuracy and robustness of SLAM systems equipped with intelligent vehicles are still difficult to meet the demand.Therefore,this dissertation takes the problem of SLAM for intelligent vehicles in complex environments as the background,exploits the complementarity between cameras and inertial navigation sensors,conducts researches on the key technologies of simultaneous positioning and mapping based on vision and inertial fusion,and designs a complete Visual Inertial Navigation System(VINS)based on a nonlinear optimization architecture to solve the problem of complex scene adaptation for intelligent vehicles.The main works are as follows.1.This dissertation investigates the current development status of intelligent vehicles at home and abroad,defines the coordinate system involved in the VINS system for intelligent vehicles,and establishes the vehicle kinematic model,camera observation model and inertial sensor motion model.Finally,it constructs the overall architecture of the VINS problem for intelligent vehicles based on the above models,and builds a research platform for the key technology researches of VINS applied to intelligent vehicles.2.The vanishing point is an important clue to build and understand the 3D world from 2D images.To solve the on-board continuous video stream vanishing point detection problem,a vanishing point estimation algorithm based on the combination of optical flow detection and edge detection is proposed.The essence of optical flow is the motion field between adjacent frames in the video stream.In the real scene,most of the vehicles travel in the direction parallel to the road boundary or lane markings,and the motion field points to the vanishing point.Therefore,the combination of edge and optical flow line to vote to the vanishing point can improve the robustness and accuracy of the algorithm.The final experimental results show that the algorithm can provide real-time and accurate vanishing point locations,which provide important 3D scene structure clues for later stop detection algorithm.3.A visual inertial guided localization algorithm that incorporates stop detection is proposed.Currently,most VINS systems assume that the environment is largely static,while the scenes surrounding intelligent vehicles are usually highly dynamic.Therefore this assumption greatly limits the application of VINS in intelligent vehicles.After an intelligent vehicle stops in a highly dynamic scene,the presence of a large number of moving feature points in the environment introduces mis-matches to the feature tracking thread,which will lead to the drift of the trajectory and the degradation of the map building accuracy,or even make the system reset.To solve the above problems,this dissertation proposes an improved phase correlation algorithm in the front end of VINS to extract the accurate motion information of adjacent image frames in the video stream.While in the back end,the stop constraint is introduced to optimize the state variables in the sliding window to solve the trajectory drift problem of intelligent vehicles during stopping in highly dynamic scenes.The experimental results show that this algorithm can greatly improve the pose accuracy of intelligent vehicles during stopping in complex scenes.4.A vision inertial navigation system based on motion constraints(VINS-Motion)is proposed.This method simplifies the motion model by using vehicle motion constraints,and fuses vision,inertial sensors and vehicle motion model to estimate poses.The system incorporates the previously proposed stop detection module in the front-end.In the backend processing,if the front-end detects that the vehicle is at a stop state,it is optimized according to the stop constraint within the sliding window.Otherwise,in addition to the residual term constructed from the existing marginalized prior information,the IMU preintegrated residual term and the visual reprojection residual term,the vehicle orientation/velocity residual is constructed according to the Ackerman steering model.The Jacobi matrix of vehicle orientation/velocity residual with respect to the variables to be optimized is furthermore derived.Finally the weighted sum of the four residual terms are minimized to obtain the maximum a posteriori estimation of the state variable.Experimental results on the dataset KITTI show that VINS-Motion has significant performance gains in challenging scenarios.5.A self-adaptive feature correspondences identification(SFCI)algorithm based on IMU information is proposed,and the SCFI algorithm is further tightly integrated into the front-end data processing unit of VINS,named VINS-SFCI.SFCI algorithm uses the results of IMU pre-integration result to predict the pose of new coming frame.In the case of weak texture and motion blur,in order to increase the number of feature matching pairs and the tracking length,it uses the pose predicted by IMU to search for potential matching pairs and merges visual information to establish new matching pairs.On the other hand,the pose predicted by IMU is also used to eliminate mismatches introduced by dynamic objects.The experimental results show that the SFCI algorithm effectively improves the feature matching accuracy and tracking length.The VINS-SFCI system realizes the accurate positioning of intelligent vehicles in complex scenes by providing more robust visual feature point tracking results.In view of the above research content,this dissertation completes the systematic verification and analysis on the dataset,and the results show that the proposed algorithms have apparent performance gain in challenging scenarios,showing important application value.The researches in this dissertation are helpful to improve the robustness and intelligence of the VINS system applied to intelligent vehicles,and have positive significance for its landing deployment in real complex scenes.