Research On Depth Video Resampling And Coding Technology

3D(Three-Dimensional) video provides visual experience for viewers with depth perception through the usage of stereoscopic displays that re-project a three-dimensional scene from the disparity of left and right eyes. MVD(Multi-view Video plus Depth) has been mainstream data representation format of 3D video. On the premise of reducing the data size of video coding and transmission, MVD can realize multi-view video system by means of DIBR(Depth Image Based Rendering) technology. Among the MVD, depth video can be provided through three methods, including depth camera, 3D geometry information from computer simulation and disparity estimation. However, the resolution of depth video which captured by depth camera is low. What’s more, the depth video obtained by disparity estimation is inaccurate and influences its coding performance. Aiming at these problems, we research on depth video resampling and coding technology combined with HVS(Human Visual System) perceptual characteristics.(1)Though the depth camera can capture depth video in real time, the resolution of captured depth video is lower than that of color camera. To address the problem, a depth upsampling algorithm based on image edge characteristics is proposed by analyzing the edge characteristics of depth image and corresponding color image. Experimental results show that the average bad pixel rate, root mean square error and PSNR(Peak Signal to Noise Ratio) of upsampled depth images of the proposed algorithm are 2.07%, 3.46 and 38.58 dB, respectively. The average PSNR of the rendered virtual viewpoints is 39.58 dB.(2)Due to the limitations of estimation algorithm of DERS(Depth Estimation Reference Software), the obtained depth video is inaccurate and exists spatial and temporal redundancy. Considering the JNDD(Just Noticeable Depth Distortion) model can describe the perceptual characteristics of human eyes to depth distortion, a depth video preprocessing algorithm based on JNDD model is proposed. Experimental results indicate that, the preprocessed depth video by proposed algorithm can reduce depth video coding bit rate by ranging from 6.23% to 42.67%. Meanwhile, it can improve the PSNR of rendered virtual viewpoints by 0.24 dB averagely.(3) For the 3D video coding of MVD format, some study suggest that people can improve the performance of 3D video coding through reducing the resolution of depth video. A 3D video coding algorithm based on depth video resampling is proposed by downsampling the depth video before encoding and upsampling the depth video after decoding under the considering the information of color video. Experimental results show that the proposed algorithm can reduce the bit rate of 3D video compression by ranging from 2.80% to 12.89% while it ensures the rendered quality of virtual viewpoint in the decoder.