Improvement Of Quality Of Depth Image Based On Non-local And Local Information

With the rapid development of imaging technology and the increasing requirement of machine vision, traditional machine vision based on color image has been unable to meet all kinds of applications based on 3D image. Compared with optical color image, depth image contains information relating to the distance of the surfaces of scene objects from a viewpoint, namely depth information. Depth image is not influenced by the reflection characteristic of the surface of an object and light condition, so it could represent the 3D information of the surface of an object more accurately. With the help of depth image, we can make up for deficiencies of traditional optical image in the characterization of 3D information ability. Currently, depth image has been widely used in the field of computer vision, pattern recognition, and human-computer interaction.Traditional methods of depth image acquisition could be mainly classified into 2 categories-the active technique and the passive technique. Currently, passive techniques are mainly based on monocular vision, binocular vision, multi-view vision and so on. The stereo matching technology, which determines the disparity of reference image based on two aligned images, is the key step in binocular vision. The matching accuracy directly affects the quality of depth image, so stereo matching has been the focus of researchers for many years. However, the accuracy of stereo matching technique currently could not achieve the criterion of practical application, a research on a stereo matching algorithm with higher accuracy is both theoretically and practically valuable. In addition, depth images captured by laser radar, TOF camera, Kinect based on structured light and other active sensors may suffer from defects such as low resolution, holes and noise due to their own shortcomings, so we need to enhance the images further in order to obtain depth images in high-quality to increase their practical value. This thesis conducts the following work based on the above two aspects and non-local and local information of depth image.1. We propose a joint framework of non-local and local pixels in the cost aggregation step of stereo matching. The method introduces the pixels of high weight based on the modified non-local means filter into the traditional aggregation method based on adaptive support weight, expanding the range of aggregation. In addition, we adopt a pixel classification and a second aggregation tactics in the refinement step of the initialdisparity image to generate the final disparity map. Experiments using the test image from Middlebury platform and the real world images show that our method could improve the estimation of the disparities of pixels near the edge of the object compared to the traditional method based on adaptive support weight. Meanwhile, the estimation of disparities of the occlusion pixels could also be improved. The quality of depth images based on binocular stereo vision has been improved.2. We propose an enhancement model for the depth image based on low-rank and second-order smoothness regularization. Combining the aligned color image with the depth image, the model takes into account the depth correlation of the non-local similar image blocks and the second-order smoothness of the local depth information. In order to solve the optimal solution of the model, we use alternating direction multiplier method(ADMM) to split the original problem into two sub-problems, and solve them alternately. Experiments using the simulation depth images and Kinect depth images show that our model can overcome the unexpected depth and pseudo texture problems compared to the methods based on traditional local filters.