The Design And Implementation Of A Real-time 3D Video System

With the development of 3D technique,the dim and low-light problem of 3D displaying have been overcame and this technique is used in everywhere now.In recent years,the virtual reality technique further improved the user experience.In this paper,a real-time 3D video system with virtual reality equipment from capturing terminal to user terminal is developed.This system includes a foreground extracting process which is widely used in the first stage in video processing.But the existing color-based foreground algorithms are easily affected by illumination changing.Besides,it is difficult for them to extract when the color of the foreground object and background are similar.A method which combines color and depth information is used to improve this process.This system can be divided into 3 modules: the stereo matching module,the foreground extracting module and the transmitting module.First,depth information is required for foreground extracting.Disparity value is calculated from a pair of left-right stereo images.Because this system is real-time required,so it uses a GPU version of semi-global matching method.After getting the disparity map,a repairing method with Mean-Shift to improve the result is proposed.Then,this paper proposes a RGB-Depth-based codebook algorithm.Depth message is insensitive to illumination,the original codebook method can be improved from two ways.First,in order to overcome the misclassification when the color of the foreground object and background are similar,a restrict is added on the range of disparity value in the code-word matching process,this can decrease the probability that a foreground object is misclassified as background.Second,for the influence of illumination and high-light region,the color distance matching restrict is broaden when the disparity value is in the range,this can decrease the probability that background pixel is misclassified as foreground object.The experiment result showed that this method can significantly improve the extracting accuracy by 10.6% in F value.At last,the processed image is compressed with H264 protocol.Through the socket way,video streaming is sent to an Android mobile phone.Then,with the virtual reality equipment,3D video of the foreground object can be accessed in real-time.