Disparity Space Based Visual Navigation For Autonomous Vehicles

In the field of ground mobile robot, autonomous navigation is a hot research topic. The study of binocular stereo vision based navigation technologies for autonomous vehicles, is very important for technology promotion and popularization of autonomous navigation. However, better utilizing binocular stereo vision sensors for environment perception and path planning still remains a challenging problem. Full mining and utilizing the characteristics of disparity space, which is constructed from binocular stereo vision, can effectively compensate for the stereo vision vulnerable to defects of image noise, scene structure and other factors, helps to promote the applications of stereo vision in autonomous navigation.This dissertation presents a novel navigation framework for autonomous vehicles equipped with stereo cameras, featuring finishing all of the perception and path planning tasks directly within the disparity space. The framework effectively solved the problem of the low reliability and efficiency of stereo vision, and promoted the applications of stereo vision in autonomous navigation systems. The main contributions of this dissertation are as follows:1. For the first time, we propose 1) the slope model in disparity space. The slope of 3D scene is modeled in disparity space, we found that, the slope computing in columns is orientation adaptive, and directly determined by the intercept of lines in disparity space, and 2) the motion model in disparity space. A new kind of coordinate for disparity point called Disparity Normalized Homogeneous Pixel coordinates, is proposed to make the motion of a rigid body has a linear form in disparity space. Thus, the motion of camera can be effectively estimated directly in disparity space.2. Based on the motion model in disparity space, linear least-squares fitting and nonlinear least-squares fitting of two disparity point sets are proposed, which effectively estimate the optimal motion directly in the disparity space. Combined with Random Sample Consensus, Extended Kalman Filter and Local Bundle Adjustment, a robust disparity space based visual odometry framework is implemented. With the estimated motion, consecutive disparity maps are fused, based on the color consistency and visibility constraint, to construct a more reliable and complete local map.3. Based on the slope model in disparity space, a novel disparity space based obstacle analysis method called V-Intercept is proposed. The method uses slope and height as two basic features to measure the cost of obstacle, Sobel operator is used for intercept computing first, then high slope region is clustered, and combined with the height of the object relative to the ground for a comprehensive cost analysis. Thus an efficient and robust analysis directly in disparity space is achieved and different cost for different types of terrain is generated. The experimental results show that the V-Intercept method is better than the classical v-disparity method, and suitable for a variety of environments.4. We propose and implement a complete disparity space based navigation framework and its structure, including an improved Semi-Global matching algorithm for initial disparity map generation, a reasonable strategy for obstacle expansion and A* algorithm based path planning and smoothing in disparity space. The experimental results show that the framework provides safe and reasonable guidance for the navigation of autonomous vehicles in the aspects of map building and obstacle analysis. Aiming at the poor real-time performance of stereo vision based navigation systems, we propose several optimization methods based on the thoughts of parallel processing, the optimized system achieved 9.5 frames per second of processing speed in 1242×375 resolution disparity images.