Research On Sub-pixel Accuracy Stereo Image Matching

Currently, Stereo vision is become one of the hot research topics of computer vision. And it has been widely applied in human-computer interaction, video surveillance, intelligent control, terrain reconstruction, robot navigation, target tracking, and other fields.In stereo vision, the most difficult problem is stereo matching, whose core issue is calculating the disparity map. And common local stereo methods are based on the integer pixel match searching step, so as to obtain integer-valued disparities, which leads to a serious jaggies on some continuous plane, especially in scenes, not perpendicular to the camera optical axis, with a large inclination plane, spherical surfaces, surfaces, etc.. Thus, to upgrade disparity accuracy from the integer-valued pixel to sub-pixel with dense disparity map become an important research direction, but also a great challenge for us. Its goal is making the disparity on the target surface naturally smooth transition and then the results of three-dimensional information recovery are consistent with the target scene. It is a significant work, especially in some fields with high precision requirements such as remote sensing, high precision3D reconstruction, medical image processing and other fields. This work conducts an in-depth research on the problem that how to increase the matching precision to sub-pixel level under the premise of high matching accuracy, and at the same time try to balance the matching speed.So far, many stereo matching algorithms have been proposed for stereo disparity measurement. And there were several algorithms that compute disparity maps with sub-pixel precision instead of integer-valued precision. In this work we proposed an improved algorithm based on a non-local cost aggregation method for stereo matching for good performance in Integer-pixel stereo matching. Firstly a high order interpolation was used on the original images to calculate cost function, and a proportional relationship between the image gray gradient of adjacent pixels and the Current pixel’s was extracted to determine the search range of disparity. And then the whole cost was Aggregated by the minimum spanning tree strategy to select the optimal fractional disparity. Finally, we got the segmentation region information of original image to do a plane fitting refinement for sub-pixel accuracy of dense matching. The experiments on different datasets show that our algorithm achieves sub-pixel precision matching.Traditional methods always assumed that all surfaces in the scene are perpendicular to the plane of camera optical axis, so widely used forward-parallel window model for stereo matching, but for some actual scenes, this assumption is not all work well. In this case, if adopting this assumption as usual, it may cause system error and wrong sub-pixel accuracy disparity. For this kind of situation, we proposed a high accuracy sub-pixel accuracy stereo matching algorithm based on slanted support window and PatchMatch algorithm. Giving up forward parallel assumption, allowing inclined surface with large angle inside the scene, we paid our attention on slanted support model. That is to say, we computed the similarity of corresponding points by estimating an individual3D plane at each pixel onto which the support region was projected and absorbing the thought of random search and propagation of PatchMatch algorithm. And then By looking for the nearest neighbor pixels who’s plane is consistent with the current pixel’s on the corresponding epipolar line, we gradually made the scope of disparity change narrowed and fund the corresponding optimal parameters of inclined plane and the best candidate pixel by iterative optimization, so as to gradually approaching sub-pixel precision matching results. Experiments are tested on the Middlebury datasets which proved the effectiveness of our algorithm.