Research On 3D Reconstruction With SFS & Stereo

Shape-from-Stereo and Shape-from-Shading are two methods of 3D reconstruction from images. Each of them has been fully studied and great progress has been made, while there are still some shortcomings and weaknesses in both. How to effectively using them to establish the 3-D model of a scene from images becomes a challenging task in computer vision research. To achieve this goal, we do some study and research on the fields of stereo matching and shape from shading in this paper.Firstly, we propose a method of stereo matching based on regions and dynamic programming. This method uses Mean-Shift algorithm to accomplish the images segmentation and detect the disparity range bound of segments. Then window matching and the reliability constraint are used to detect ground control points. And the disparity consistency correlation between GCPs and segments which GCPs pertain to is used to remove some suspicious GCPs. At last, we use the ground control points and segments as constraints and modify the data term and the occlude term with segmentation information to compute the disparity map using DP algorithm. Experimental results show this method can get a good performance.Secondly, we propose a method of constructing the SFS global feature transform to assist the pixel-to-pixel matching in stereo matching. After theoretical proof and experimental validation, we verify that the disparity results of shape from shading could be a global feature of stereo matching. Based on this, we construct the SFS global feature vector with the disparity images obtained by the methods of shape from shading and use them to help the stereo matching. The experimental results on Middlebury test set indicate that our method improves the accuracy of stereo matching and obtains a well performing disparity results.