A Study Of Kinect Data Repair Method And Application On Stereo Video

In modern times, the emergence of remote video greatly facilitates the communications among people, and the development of three-dimensional visual makes video communication more promising. Depth acquisition becomes inexpensive after the revolutionary invention of Kinect. It’s a practical approach to construct three-dimensional model data from Kinect, and achieving stereoscopic video conferrencing system.But for computer vision applications, depth map captured by Kinect require additional processing to fill up missing parts. In the reconstruction process of three-dimensional model with Kinect data, missing color data needs to be repaired before final display due to the perspective transform. This paper firstly studies the method of Kinect to build threedimensional data, then fixes the missing of various reasons for three-dimensional data with structure-oriented framework to repair the depth data and Fast Marching Method to repair the color data, finally designs a conference system of three-dimensional video and applies the repair algorithm on it, at the same time studies a variety of acceleration strategies to improve the playable frame rate of the stereoscopic video.While repairing the depth data, conventional inpainting methods for color images cannot be applied directly to depth maps as there are not enough cues to make accurate inference about scene structures. In this paper, we propose a novel fusion based inpainting method to improve depth maps. The proposed fusion strategy integrates conventional inpainting with the recently developed non-local filtering scheme. In choosing the reference point, we propose a selection method which has space in eight directions, which may reflect the color and depth characteristics around the point to be required well. The good balance among distance, depth and color information guarantees an accurate inpainting result. Experimental results confirms the effectiveness of the proposed method whether from the subjective evaluation or the quantitative accuracy analysis.One typical application of Stereo Vision is the remote video conference system. This paper designs and implements a system with split-screen 3D TV and Kinect device for remote video conferencing after repairing the three-dimensional data. What the system needs to solve at first is to design stable and efficient device driver modules which should provide stable data streams to upper modules on different platforms at the same format, ant Open NI possesses such characteristics. Multithreading is indispensable to make both background restoration algorithm and front end interface operate smoothly. This paper adopts circular buffer of fixed size to achieve data sharing among various threads of each module. Actual operating results prove that this high-speed buffer module can balance common thread safety issues in multi-threaded programming and operational efficiency well. In order to improve the fluency of video conferencing system various methods need to be adopted to accelerate the calculation process. This paper uses a multi-threaded CPU to accelerate the repair algorithm and CUDA custom kernel in GPU to accelerate the reconstruction of the model. Experimental results confirms that both of the acceleration method can significantly improve the speed of repairing depth data and the frame rate of stereoscopic video to display.