Stereo Matching And Depth Enhancement In Image-based Depth Perception

Image-based depth perception, which aims at recovering depth information from single or multiple images, is a key research problem in computer vision area. It has many applications including 3D modeling, robot navigation, street view and up-to-date mobile consumer products. The primary building blocks of image-based depth perception are stereo matching and depth enhancement. Stereo matching is the problem of estimating pixel-level dense correspondence between two images and depth enhancement is the problem of improving depth information quality, where depth information comes from various depth perception techniques. In new application scenarios, there are many difficulties and challenges in stereo matching and depth enhancement research. In this paper, to solve the noise, illumination variation and textureless problems in stereo matching, we propose a new stereo matching algorithm and an algorithm framework to improve stereo matching accuracy; to solve the unified modeling problem of depth enhancement, we propose a twostep approach to emphasize the importance of confidence measure in depth enhancement.To be specific, the main contributions of this paper are:? We propose a binary stereo matching algorithm. We introduce binary feature descriptor into stereo matching. To solve the edge fattening problem, we invent a self-adaptive pixel pair sampling method based on the assumption that color similarity can be used to approximate disparity similarity. The self-adaptive sampling method is implemented by binary mask so that matching cost can be computed by bitwise operations. In the disparity refinement process, we propose a weighted voting method to improve the matching accuracy. Experiments on public datasets show that the proposed algorithm is accurate, efficient, and robust to illumination and can be adjusted according to different requirements.? We propose a multiscale stereo matching framework. The cost aggregation step of stereo matching can be modeled as a weighted least square problem. We use this model to analyze state-of-the-arts cost aggregation algorithms. To introduce multiscale information into cost aggregation, we extend the weighted least square problem into scale space and add a multiscale regularization term into the optimization objective. Various cost aggregation algorithms can be integrated into our multiscale cost aggregation model resulting in a stereo matching algorithm framework. Exper-imental comparisons and analysis on public datasets prove that our framework can improve state-of-the-arts algorithms’ accuracy.? We propose a confidence-based depth enhancement algorithm. We propose to model the single-view depth confidence estimation problem as a bi-label markov random field so that we can get depth confidence efficiently. Based on the depth confidence, we propose a confidence-based searching algorithm to refine depth.Comparing to conventional joint filtering methods, the searching algorithm can handle large errors in local region. Experiments in depth superresolution and denoising show that our method can produce more accurate depth information.