Research On Mobile Robot Binocular Stereo Vision

This paper based on Marr vision theory has studied the technology of binocular stereo vision of the mobile robot. The aim of the research is to provide for the robot the depth and 3D coordinates of interesting object in the environment from the images captured by the left and right cameras on the robot, while the robot are moving.This research focuses on three components of binocular stereo vision: camera calibration, stereo matching and 3D information calculation. In camera calibration, Zhang's calibration method is adopted in the first stage to get the inner and exterior parameter. In the second stage we can get the motion parameters of cameras by self-calibration technique which is based on fundamental matrix and essential matrix, and at the same time supposes the inner parameter is invariable when cameras are moving. In stereo matching, this paper studied two algorithms, one is area-based matching algorithm, the other is feature-based matching algorithm. Area-based algorithm takes maximal intensity cross-correlation of two windows in left and right images as the matching measure, at the same time makes multiple judgements considering some vision restriction, to improve precision and match speed. Feature-based algorithm adopts Harris Corner to extract feature points at first. After correlate matching, we get a rough matching points. There are inevitably many false matching pairs. But we can get the fundamental matrix from this matching pairs by LmedS robustly. Then the epipolar geometry of stereo vision described by fundamental matrix is used to eliminate the false matching pairs and get more precise matching points. After the camera models and the matching points in two images are gained, the coordinates of the point can easily be calculated, which is the resolution of some projection equations. This can be achieved by least square algorithm.This thesis has studied the crucial techniques of binocular stereo vision, including image filtration, feature point extraction, camera calibration, the estimation of fundamental matrix, stereo matching, the calculation of 3D coordinates. The results of experiment with real images show that the methods used in this article is correct and feasible. This work provides a foundation for developing binocular stereo vision on mobile robot.