Research On Some Key Theoretical Issues For Robotic Binocular Stereo Vision And Its Techniques Implementation

Binocular stereo vision is an important part of computer vision, which employs two different location cameras or one camera move and rotation to shoot the same scene, then calculating the parallax of the corresponding points in two images of spatial point. After a series of inverse projection transformations, the 3D coordinates of spacial point can be obtained. Compared to other types of stereo vision methods, such as holography,3D imaging lens board and so on, binocular stereo vision directly simulates the ways of the human eyes processing the scenes. Therefore, binocular stereo vision is a simple and reliable approach to acquire the information of real world. Binocular stereo vision involves many challenging problems in the field of Artificial Intelligence, Computer Graphics and Cognitive Psychology, etc. Studies deep into it have been carried out abroad and technologies of binocular stereo vision have been widely used in people's lives. However, in our country, research on binocular stereo vision is in an initial stage, vast scientific and technological work needs to be done for more contribution.Consequently, in order to find a solution to several key problems of the binocular stereo vision, the dissertation focuses on related procedures of implementation of the binocular stereo vision, aims at solving some core theories and key issues of constraining the development of binocular stereo vision, combines theoretical studies and experimental validation, basic configurations and principles of the binocular stereo vision system are investigated profoundly, involving stereo vision cameras calibration methods and stereo matching algorithms. This paper is divided into seven chapters and each of them is arranged as follows:In chapter I, the development of computer vision theory research is reviewed, and the application status of the binocular stereo vision techniques at home and abroad are investigated. Then, taking the stereo vision as the key point, the application prospects of binocular stereo vision are analyzed. Then the key techniques and existed issues of related research are discussed. Finally, the significance, difficulties and main content of this research topic are clarified.In chapter II, firstly, a brief introduction to linear and nonlinear camera perspective projection model, and then the imaging principle and mathematical model of the binocular stereo vision are stated in detail. Meanwhile, this paper elaborates the basic concepts of epipolar geometry and its rectification process. Finally, the commonly used constraints in stereo matching implementations are deeply discussed.In chapter III, in order to solve the calibration problem of the binocular stereo vision cameras, firstly, this paper introduces standard calibration methods of single camera based on 3D target and 2D planar template. In the mean time, the calibration principle of the binocular stereo vision cameras is studied. Then, the camera model in the Open Source Computer Vision Library (OpenCV) is discussed, a profound investigation of the camera calibration process is performed. Specially, the radial distortion and tangential distortion had been deeply taken into account. Finally, experiments are implemented on four different kinds of checkboard images. According to the experimental results, it can be concluded that the greater number of the checkboard, the higher precision of the camera calibration. The calibration precision of the asymmetrical checkboard is better than the symmetrical one if the size of the checkboard is the same. Meanwhile, the entire calibration process of the former does not need human involvement and is fast. The calibration results meet the calibration accuracy of practical applications.In chapter IV, firstly, the basic concepts and the general procedures, such as matching cost computation, cost aggregation, etc, are introduced. Then, the implementation of traditional non-parametric transforms stereo matching is studied, and their limitations are analyzed. After that, gray values of the pixels in the transform window are averaged, and the mean value is taken as the gray value of the center pixel. Meanwhile, in order to take the mutual information into consideration while finding stereo correspondences, the original gray values of the neighborhood pixels whose relative position is one unit greater than the center pixel are replaced by the gray value through bilinear interpolation. The non-parametric transforms of the gray values of the pixels in the transform window are implemented before stereo matching performed. Finally, with the stereo matching experiments of four standard data sets, experimental results fed back by Middlebury illustrate that compared with other single local matching algorithms, the percentage of bad matching pixels of the proposed algorithm is nearly equivalent to other algorithms. In addition, with the stereo matching of the scene in real-life conditions, it also gets satisfying dense disparity map in the textureless regions, the occluded regions and depth discontinuity regions. What is more, the results of stereo matching under non-ideal lighting conditions are good.In chapter V, aiming at the stereo matching with complicated backgrounds, firstly,3D vision in robotic applications and the commonly used dissimilarity measure functions based on local stereo matching are introduced. Then, to evaluate the stereo matching performance of those dissimilarity measure functions, two standard image sets and two real world images are used respectively. The quality of the disparity map and the best dissimilarity measure function which has the least execution time are obtained. Meanwhile, for those areas with high error rates of the complex background images, the left-right consistency filter, the confidence filter, and the uniqueness filter are applied to eliminate the bad matching points. Finally, the configuration of the stereo matching, the memory organization and high speed memory, are optimized. Furthermore, the SSE2 (Streaming SIMD Extensions 2) Multimedia extensions commands are adopted to accelerate the execution speed of the stereo matching. Experimental results show that after taking these precautions, the reliability and real time of the stereo matching of the images with complex background have been greatly improved.In chapter VI, firstly, the principle, methods, and configuration of 3D reconstruction is described respectively. Then, the primitive objects 3D reconstruction, especially, the reconstruction of 3D points is focused on. Finally, based on the contents proposed in chapter III, IV and V, the images of 3D reconstruction of special points for complex background is implemented. And with the tool of OpenGL, the dense 3D points cloud is displayed. Meanwhile, the distance between the closest point and the left camera is computed, which validates the comprehensive application of the algorithms proposed in the dissertation.In chapter VII, the conclusions of the research are summarized, the innovations in this dissertation are pointed out, and suggestions are provided for the following related research.