Robust Stereo Matching Algorithm Based On Ground Control Points And Energy Optimization

Stereovision is a method of restoring 3-D information from two or more 2-D images of the same scene. During its 30 years' development, many achievements in research and application have been made. Some special stereovision systems are widely used in fields of industry, military, medicine, etc. But on the whole, stereovision, especially general stereovision systems, is still a widely concerned research field with lots of difficulties. One of the most critical problems is stereo matching, and are also called stereo correspondence.Stereo matching needs to obtain 3-D depth information from 2-D image information, therefore it's a ill-posed problem. Certain difficulties remain, for example, the matching uncertainty and occlusion. And most existing stereo matching algorithms have insufficiencies of various degrees in real-time application, reliability, robustness, universality, etc. Therefore, under the guidance of ground control points (GCP), a new stereo matching algorithm with high reliability is adopted in this dissertation. It is based on region segmentation and energy optimization, and combined a stepwise iterative strategy.The main components and innovation points of the dissertation include:(1) GCPs play a conducting role in this algorithm. They not only provide foreground and background information for region segmentation, but also are the major sources of the disparities of regions. In this algorithm, lots of constraints are used to ensure the high reliability of GCPs, which offers a good starting point for the procession. In addition, a method based on gradient is adopted so that to control the quantity of GCPs and reinforce their reliability to a certain degree.(2) It is one type of region-based algorithm. This kind of algorithm uses "region" as the base element of matching and other processes, so it usually has better effect on disparities discontinuity and occlusion problems. However, the accuracy of region segmentation strongly influences the final result. Inadequate segmentation causes unclear surfaces. As a result of over-segmentation, the change of disparities in regions can't be reflected in results. So, a graph-cuts-based iterative region segmentation algorithm under the guidance of GCP is applied. Firstly, a common image segmentation algorithm is used to obtain initial regions, then, a K-mean GCP clustering process is carried out on each region. For those regions which have two or more clusters, a graph-cuts-based algorithm is adopted for re-segmentation. Then execute the above procedure iteratively until every region has no more than 1 GCP cluster. Finally each GCP cluster stands for certain range of disparity, and the result of regions can correctly reflect the distribution of surfaces in the scene. Meanwhile, the interior GCP clustering procedure reinforces the reliability of GCP: some outliers of GCPs are excluded.(3) The dense disparity distribution in regions can usually be obtained through interpolation, plane or curve surface fitting methods from sparse disparities (by means of window-based correlation matching, block matching, feature-based matching, etc.). Due to the complexity and noise sensibility of fitting methods, a single disparity is usually used to represent a region in research. However, this method has several disadvantages. For example, the precondition of the fact that a single disparity can stand for a region is over-segmentation, which will create more regions and cause larger time and space consumption. If there are lots of curve surfaces or inclined planes in the scene, single disparities can't reflect the actual disparity distribution correctly. Therefore, a method of constructing Delaunay-triangular mesh based on discrete disparities is applied to the interpolation of region disparities: Firstly, obtain the range of region disparities from the result of interior GCP clustering process, and use a dynamic programming algorithm to receive disparity distribution on region boundary under constraints of disparity range, disparity continuity, etc. Secondly, select some edge points and interior GCPs to construct Delaunay-triangular mesh. Finally, use linear interpolation on each triangular plate so as to obtain dense disparity distribution on region. This approach avoids the complexity and high consumption of curve surface fitting. Meanwhile, the disparity distribution result would reflect the change of disparities on surface to a certain degree, and the result precision of sub-pixel is achieved.(4) In order to ensure the accuracy of region disparities, in this algorithm, the "best disparity" of each region is obtained through a method of energy optimization. After the definition of energy functions on region, the "best disparity" is the one that achieves the minimum energy in candidate disparities from the adjacent regions and interior GCPs. For the "matched region" which has a GCP cluster, the "best disparity" is used to prevent its disparities from GCP's matching errors; for the "unmatched region" which hasn't any GCP cluster, the "best disparity" is used as its single disparity which can be seen as a correct disparity spread from adjacent regions. In order to reinforce the reliability of disparity propagation, some "reliability criteria" is applied to decide the processing sequence of regions besides the 2-step strategy ("matched region" first, then "unmatched region"). For each region, a measure of "reliability" is computed according to some reliability criteria, and the region with higher "reliability" has processing priority. The algorithm uses several "rounds" of processing as below: Firstly, a higher threshold of reliability is set, and only those regions whose "reliabilities" are higher than the threshold will be handled in this round. Then, the threshold decreases as the round number increases. Finally, all the regions are handled. This strategy ensures that the disparity with higher reliability can be spread out through neighborhood more easily, so the final global disparity result will be more accurate.In conclusion, by the method of GCP, energy optimization, stepwise iteration and so on, our algorithm can gain higher result precision of disparity. The experimental results show that it's a fast and reliable stereo matching algorithm.