Wide-baseline stereo for three-dimensional urban scenes

Like humans, computer vision systems can better infer a scene's 3-D structure by processing its 2-D images taken from multiple viewpoints. While this seems effortless for humans, it is still a challenge for computer vision. Underlying the act of associating the different perspectives is a problem called wide-baseline stereo, which computes the geometric relationship between two overlapping views. Wide-baseline stereo can be problematic when working on images taken of real-life urban environments, due to practical issues such as poor image quality or ambiguity raised by repetitive patterns. We analyze why these factors pose difficulties for current methods and propose principles that can make wide-baseline stereo more effective, in terms of both robustness and accuracy.;In the case of feature matching, we show that we can match features more robustly when using both local feature appearance and regional image information. We model global image information with a graph, whose nodes contain local feature appearances and edges encode semi-local proximity structure. Working on this graph, we convert traditional feature matching into a graph-matching problem|essentially, we are shifting from a purely local to a context-driven feature matching paradigm. In comparison against local methods, our algorithm performs robustly and is consistently better under difficult wide-baseline conditions, such as repetitive local patterns, under excessive image noise or low resolution inputs.;For the fundamental matrix estimation, we propose to implement a preprocessing step on the feature correspondences before commencing the estimation procedure. This is essentially a registration-based re-alignment on correspondences, where we locally adjust the position and shape of the feature in one image according to the appearance of its match in the other. Our experiments show that the preprocessing consistently increases efficiency and accuracy of the fundamental matrix estimation.;In summary, we propose a series of algorithms for wide-baseline stereo. Essentially, our methods achieve better robustness and accuracy than current approaches by making use of more image information. By combining entropy-based saliency with intensity contrast, our feature detector is better than its peers at detecting regular man-made structures in the presence of unwanted high frequency patterns regarded as noise. By using neighborhood information, our feature matching method is less sensitive to appearance ambiguity than traditional matching methods. The preprocessing step exploits information contained in both images to refine localization of matched features. These techniques can be especially useful for practical 3-D vision applications, for example, to robustly model or render a 3-D scene based on less-than-ideal input images taken of real-life environments.;We treat wide-baseline stereo as a sequence of three sub-problems: feature detection, feature matching, and fundamental matrix estimation. We propose improvements for each of these and test them on real images of 3-D urban scenes. For feature detection, we demonstrate that when we use both image intensity contrast and entropy-based visual saliency, we are better at repeatably extracting features of a 3-D scene. We use intensity contrast as a cue for obtaining initial feature seeds, which are then evaluated and locally adapted according to an entropy-based saliency measure. We select features with high saliency scores. Experimental comparisons against peer feature detectors show that our method detects more regular structures and fewer noisy patterns. As a result, our method detects features with high repeatability, which is conducive to the subsequent feature matching.