Research On Depth Enhancement Problem Based On RGB - D Image

A new inexpensive RGB-D camera like Microsoft Kinect is a somatosensory sensor which has been widely applied in animation, comics and games. There is a great deal of noise in depth image from RGB-D camera, so how to obtain a detailed, smooth and acceptable 3D model is an acknowleged thorny problem by the coarse and even incomplete information.Under the support of the Science and Technology Development Programme of Shandong Province "Reach and development for the portable 3D body measurement system based on depth camera", this paper mainly focuses on the problem of the depth map enhancement. The main contents and contributions of this thesis are as follows:(1) In this paper, Zhengyou Zhang’s calibration method was applied to accomplish the camera calibration of RGB camera and depth camera. Since the data captured from the two cameras have slightly different view points, we need to register them.(2) A new algorithm to generate the high quality depth map was proposed. First, the smoothing filter templates and sharpening filter templates were used to process the RGB image. At the same time, 3D warping was performed to pre-process the depth map. In consideration of the depth map is unstable in the time domain. We use histogram statistics to update the frame depth value based on the temporal correlation and we proposed a new method of adjacent frame difference method to reduce the computing time. And then, we fill the hole based on the spatial correlation of the depth map. We construct a quadratic objective function to propagate sparse depth map values to the high resolution RGB image under the assumption of there are co-occurrences of depth boundaries and images. Then, we adopt the method of bilinear interpolation upsampling the depth map. At last, the complex median filtering was used to smooth the depth map and to eliminate the influence of the outlier data. Experimental results show that proposed method not only maintains fine detail and structure, but also fills the holes and improves the resolution of the depth map so that we can effectively generate the high quality depth map.(3) We create a 3D reconstruction with the program of Kinect Fusion and high quality depth map. The surface reconstruction technology by PCL was applied to smooth the 3D reconstruction model and the poisson equation was proposed to remove the noises. Finally, the entire and high precision surface was reconstructed.