Research On Video Resampling For Multi-view Plus Depth Coding

Multi-view plus depth video is a popular 3D video format,which consist of texture video and depth video.Multiple virtual views can be attained by view synthesis.Multi-view video provides multiple perspectives for the stereoscopic scene,Stereo perception and interactive function on the sense lead to promising 3D video application,However,the transmission of the texture and depth of the multiplexed video signal will bring a problem that video storage and transmission needs large volumes of data.How to achieve efficient multi-view coding is a key issue in the current 3D video coding.When the video resolution is reduced before encoding,then upsampled to original resolution after decoding,it becomes feasible to improve video compression efficiency.The amount of data are reduced by downsampling at the encoder,so as to decrease the coding rate.On the decoder,it is upsampled to the original resolution and reduced the distortion of synthesized view by video resampling.In this paper,code rate reduction by video resampling and quality improvement by synthesized view are studied.It contains three aspects:At first,considering that only depth map is downsampled by the existing multi-resolution3D-HEVC encoding,which is not referred to the coding structure of 3D-HEVC video.We proposed a mixed resolution video coding method targeting low bit rate application,where base view is coded at the original resolution to ensure the quality of the base view and dependent view is coded at reduced resolution to reduce bit rate further.And the median down-sampling based on the mean division to enhance coding efficiency.Considering the sensitivity of the human eye for luminance information is higher than the chrominance information,we proposed a double interpolation filter of the upsampling method,where the 6-tap filter is performed on the luminance component for accurate upsampling,while the 4-tap filter is performed on the chrominance upsampling to reduce complexity.Experimental results show that it can effectively save the bit rate.Secondly,guiding filtering is applied for video preprocessing in order to protect the edges of object before depth downsampling.We proposed the method which the depth map is filtered by the guidance image which consists of luminance and chrominance.The experimental results show that our method attains higher coding efficiency and better quality of synthesized view than the method using the weighted chrominance component or the weighted luma and chrominance component of the texture as the guidance image.Thirdly,because the object edge of synthesized view by the existing depth resampling methods often causes distortion,A depth resampling method based on neighborhood filtering forview synthesis is proposed in this paper.The pixels in the sampling window are divided into two groups by the mean at first.The pixel group larger than half number of pixels in the window is selected as candidate group,where the downsampled depth is determined by the similarity of the defined characteristic depth value and centre pixel value.For depth upsampling,the common edges between texture image and depth image are found.The intensity of adjacent pixel around common edge is calculated,which is related to spatial similarity,depth similarity and texture similarity.The pixel pair whose intensity difference is smallest among four pixel pairs with horizontal,vertical,450,1350 direction,is used to estimate the direction of edge whose depth is renewed by the mean of pixel pair.In addition,the depth of non-edge pixel is refined by median filtering.Experimental results demonstrate that the proposed approaches can improve the coding efficiency and synthesized view's PSNR as well as the good visual quality.