Study On Depth Image Enhancement Algorithm For Kinect Sensor

With the rise of Kinect sensor, depth image processing is rapidly becoming a hottopic in the field of image processing and computer vision. Applications based onKinect are also booming. However, limited by the principle of the sensor, the depthimage from Kinect suffers the following four kinds of problems: noise proportional tothe distance square, holes caused by invalid pixels, unmatched edges between depthimage and its corresponding color image, and flicking in the temporal domain. Thesefactors hinder the usage of Kinect in the field such as face recognition and3Dreconstruction where highly accurate depth data is required. To improve the quality ofKinect depth image, the algorithms of Kinect depth image enhancement in differentconditions are investigated. Below are the main works:1. A single depth image enhancement algorithm. After investigating into depthmeasurement principle of Kinect sensor and analyzing characteristics of Kinect depthimage, we discover that the unmatched edges problem is caused by wrong pixels inthe depth image between depth edges and their corresponding color edges. Based onthis discovery, a wrong pixel removal algorithm is proposed using region growing anddistance transform. After wrong pixel removal, a weighted mode filtering is used tofill holes. To reduce noise in depth image, an adaptive joint bilateral filter is adopted.Experimental results show that the proposed method outperforms the state-of-the-artmethods.2. An enhancement algorithm for static scene depth sequence. In order to benefitfrom information of temporal domain, the proposed single depth image enhancementmethod is improved in two ways. The first one is to estimate the depth values forsome pixels of holes before using weighted mode filtering. To this end, a depth mapmodel and an invalid pixel model are established。The values of pixels fitting theinvalid pixel model are estimated by the depth map model. The second one is toeliminate flicking by using Kalman Filter. According to the characteristics of thedepth image, the variance of noise in the observation equation is adjusted adaptivelywith the depth value.Besides the two algorithms above, a comprehensive review of the existingtechniques for depth image enhancement is presented. Since depth imageenhancement is a relative new topic, no review on this topic is made to the best of our knowledge. So the review presented in this paper is also the result of our effort andcan be quite instructive for researchers interested in depth image processing.