The Research On Kinect Depth Image Inpainting Technique

The kinect has attracted attention of many scholars since it came out in November,2010. It has been applied to many computer vision fields so far, especially in the tracking and recognition fields. However, few research on matching and segmentation technology based on kinect are proposed. The reason that limits its application in these fields is the effect of the kinect’s various errors. Therefore, depth image inpainting technology of kinect will receive more attention in order to expand the applications of kinect in the future. This paper discusses the depth inpainting technology from the point of view of single-frame and multi-frame. The main works are as follows:In the case of single frame, this paper uses color image’s information to refine the depth image. Although the kinect is able to generate depth images and color images at the same time, the objects’boundary is not aligned between the two images. The depth values of the object in the depth images will overflow the object boundary. These values are called interference depth values in this paper. In the view of this situation, this paper proposes an interference cancellation method. It uses canny algorithm to detect the edges of the depth image and color image. Then the pixels between the edges of two images will be set to black hole (that is to say to set the gray value to zero). Finally we can get a depth image after interference cancellation.Considering the depth image after boundary alignment will have a large area of black hole, this paper combines with the thought of iteration. This paper proposes an iterative joint trilateral filter under the consideration of spatial information, color information and structural similarity (SSIM) of the color image to carry out black hole filling. This joint trilateral filter is used to fill the black hole area to obtain the high quality depth image. After filtering there will be artifacts in the depth image. In order to eliminate the artifacts without destroying the boundaries in the depth image, an adaptive processing range filter is implemented in the trilateral filter in this paper. And then filter the depth image with it to eliminate the artifacts.In the case of multiple frames, due to the random errors and precision of kinect will change with the distance between target and kinect, it will cause random jumpy-changing of depth values of the object surface when the object is a little far from the kinect. In order to solve this problem, this paper establishes a relationship between various depth images by kalman filter algorithm. It combines the predict depth value and the measurement depth value in the current moment to predict optimal current depth value. The experiment shows that the method proposed in this paper can not only obtain a high quality depth image, but also solve problem of depth values jumpy-changing with the time.