Research On 3D Scene Reconstruction From Binocular Stereo Pairs

In the field of computer vision, the 3D reconstruction is a very important issue and binocular stereo matching is a classic method of 3D reconstruction. The matching algorithm compares two photos of the same scene which are taken from different positions to find out corresponding pixel pairs. Different distances of these corresponding pixel pairs are used to calculate the relative distance (depth) from scene points to camera. Thus 3D scene information can be reconstructed from 2D images. Binocular stereo matching is widely used in many areas such as virtual reality, cultural relic protection, industrial monitor, digital entertainment, etc. Binocular stereo matching requires lower level of equipments, time complexity and space complexity of algorithm comparing to other 3D reconstruction methods. So it has important research value and application potential.Based on taxonomy, analysis and comparison of binocular stereo matching, we find there are some research trends and important topics: more global, efficiency, robust to noise and universal property of algorithm. This thesis focuses on investigation of research situation in this area and implementing, comparing and improving several classic methods which have good results. Then we select and implement a new binocular stereo matching method which is accurate, efficient and robust to noise, and meets the application demands.Recently there are many efficient methods in the area of binocular stereo matching. We select and implement three typical and top rank methods. They are Belief propagation based symmetric model, semi-global stereo matching and image segmentation based mean-shift stereo matching. After evaluating and comparing results and efficiency of these algorithms, we finally select semi-global stereo matching as the base algorithm to be improved in this thesis. The enhanced algorithm yields relative good results, and it has a great advantage on efficiency. Besides, it can be combined with the technique of single instruction multiple data of modern CPU which can accelerate the algorithm greatly. Semi-global stereo matching is also robust to noise.In second part of this thesis, we make several improvements to solve problems in semi-global stereo matching algorithm which are also main contribution of this thesis.1. To solve the mismatch problem caused by incomplete path, we add a soft structure constraint into the process of matching. Structure information of scene is extracted by image segmentation and plane model parameters estimation using least square method. Then this structure information is added into pixelwise matching cost calculation as soft constraint.2. We also use structure information to give a reliable estimation of occlusion and mismatch area.3. In order to solve the smooth problem within segment and avoid blur of depth boundary, this thesis analyses and modifies the process of fitting and propose a fast quadric surface fitting algorithm combined with structure information.Experiments demonstrate that the performance is improved significantly by solving depth pollution caused by incomplete path, and providing reliable estimates for occlusion and mismatch. Besides, our algorithm reserves efficiency and ability to speed up using SIMD by retaining the framework of Semi-Global algorithm and has extensive practical value.