Point, line segment, and region-based stereo matching for mobile robotics

At the heart of every stereo vision algorithm is a solution to the matching problem - the problem of finding points in the right and left image that correspond to a single point in the real world. Applying assumptions regarding the epipolar rectification and color similarity between two frames is often not possible for real-world image capture systems, like those used in urban search and rescue robots. More flexible and robust feature descriptors are necessary to operate under harsh real world conditions. This thesis compares accuracy of disparity images generated using local features including points, line segments, and regions, as well as a global framework implemented using loopy belief propagation. This thesis will introduce two new algorithms for stereo matching using line segments and regions, as well as several support structures that optimize the algorithms performance and accuracy. Since few complete frameworks exist for line segment and region features, new algorithms that were developed during the research for this thesis will be outlined and evaluated. The comparison includes quantitative evaluation using the Middlebury stereo image pairs and qualitative evaluation using images from a less structured environment. Since this evaluation is grounded in urban search and rescue robotics, processing time is a significant constraint which will be evaluated for each algorithm. This thesis will show that line segment-based stereo vision with a gradient descriptor achieves at least a 10% better accuracy than all other methods used in this evaluation while maintaining the low runtime associated with local feature based stereo vision.