Stereo Matching For Three Dimensional Visual Communication

Stereo vision is a fundamental topic in computer vision. There have received a breakthrough on multiple view geometry in the past decades. At the same time, people described disparity field as Markov Random Field, formulated stereo matching as pixel labelling problem, applied graph cut algorithms or belief propagation algorithm to estimate the disparity field, and got very good experimental results. In recent years, three dimensional visual communication and image based rendering etc. are becoming new applications of stereo vision and require both high quality and high efficiency of the matching. In this thesis, we develop some novel approaches for stereo matching to meet these requirements. The main contributions consist of:1. We propose bisection approach for pixel labelling. It assigns the whole label set to each pixel at first, splits the label set into two subsets and discards the one with higher cost of assigning it to the pixel iteratively, until each subset contains only one label. We present a probabilistic interpretation of the process, construct an energy function to optimize it, and prove that the constructed energy can be mini-mized via graph cut exactly. Based on bisection approach, we propose bit setting algorithm, it sets one bit of each pixel's label at each step. Bit setting algorithm has complexity of (log2n), is most efficient among state of the art techniques. We apply bit setting algorithm to solve stereo correspondence problem. Exper-imental results demonstrate that both good performance and high efficiency are achieved.2. We propose bilayer stereo matching for scenes consist of foreground and back-ground. It first determines disparity fields for foreground layer and background layer independently, then combines them together to get the final disparity field. Unlike previous layered approach for stereo matching, it does not need model fitting and iterative adjustment. We also make use of color information and con-trast information in one image to determine a better segmentation of foreground and background. Experimental results demonstrate that bilayer stereo matching improves precision greatly, has advantages on both quality and efficiency over Layered Dynamic Programming. 3. Based on above approaches, we present some solutions for gaze correction and foreground/background segmentation, which is necessary for three dimensional visual communication. We proposed a view synthesis algorithm based on bilay-ered description of scene, propose a foreground/background segmentation algo-rithm based on bisection approach for pixel labelling. More experimental results demonstrate that the techniques proposed in this thesis are effective.