An Algorithm For Depth Image Enhancement Based On Planar Constraints

In recent years, with the depth camera technology being more and more mature, the depth images have been widely used in practice. Because each pixel on the depth images has not only color value, but also the corresponding depth value, which provides a new perspective and tool to solve the hard problem in the fields of computer vision and image processing, brings about revolutionary changes in these research areas.However, there are some defects on the depth images generated by Kinect and other depth cameras. Some areas on these depth images missed information, and accompanied by a lot of noise. To obtain depth images of higher quality, we must pre-process these raw depth images, which provide a good foundation and help for the later processing.In this paper, we first summarize the two common depth image enhancement algorithms, one is based on the filtering method, and the other is based on the energy function method. And we compare the advantages and disadvantages between them. On the basis of these, this paper introduces the prior knowledge of plane constraint. Because most of the depth images’application scene is mostly indoor scenes which exist a large number of objects that are made of planes, such as wall, ground and desktop.Two different models are proposed in this paper, which are based on the prior knowledge of planar constraints that added into the existing algorithms.The first model is a depth image enhancement model with planar line constraints. The model assumes that the depth of each center pixel is close to the depth of the line in the neighborhood of each center pixel. After adding it to the model, a quadratic energy function is obtained, and the only closed solution can be obtained by solving a sparse linear equation group. The experimental results show that the first model has a better effect than the previous two common depth image enhancement algorithms.The second model is a depth image enhancement model with planar fitting constraints. The model assumes that the depth values in the neighborhood of each pixel should be as close as possible to certain optimal plane. This optimal plane is be obtained by fitting the depth values of all pixels in the neighborhood. There are two different types of unknown variables in the energy function in the second model, and no exact solution can obtained to the problem. In this paper, we will minimize the energy function of the second model, and decompose it into two sub problems which are easy to solve. And the two sub problems can be solved by alternately, and the approximate solution of the second model can be obtained. The experimental results show that, whether in objective numerical comparisons, or subjective visual comparisons, the experimental results of the second model have been significantly improved, and achieve more satisfactory results than the previous algorithms.Finally, this paper summarizes the two models with planar constraints, and points out the possible direction of the future research.