Research On Multi-View Plus Depth 3D Video Error Concealment Technology

Three-dimensional(3D)video can provide more fidelity and immersion to the audience,and are gaining increasing amounts of acceptance and jubilation.Multi-view video plus depth is the main encoding format adapted by the 3D video encoding based on the high efficiency video coding(3D-HEVC).It uses virtual view synthesis technology at the decoder to extended the amount of viewpoint from the limited number of compressed viewpoint information.Due to network congestion and noise interference,some data packages may be lost during transmission of the compressed multi-view video streams.If the multi-view video has an error or a packet loss during transmission,not only the decoded images in same viewpoint have different degrees of distortion,but also the rendering images produced by the viewpoint synthesis technology will be degraded.Therefore,it is indispensable to perform error control for multi-view plus depth video,and error concealment at the decoder is the core issue for error control technique.How to take full advantage of the temporal,spatial and inter-view correlation to get more accurate motion and disparity vectors?How to better reconstruct images with those motion or disparity vectors?And how to reconstruct the entire texture image with depth map?All of those questions are need to be solved.Based on the analysis of key technology of the 3D-HEVC and virtual viewpoint rendering technology,this thesis proposes two novel error concealment algorithms for multi-view plus depth video encoding by the 3D-HEVC standard.The main contribution of this work includes:1.Based on the correlation between the viewpoints of multi-view video sequences,combined with the inter-view motion vector sharing technology,an error concealment algorithm for deep sequence transmission loss in 3D-HEVC coding is proposed.First,based on the 3D-HEVC hierarchical B-frame prediction structure and the depth map texture feature,the depth map lost block is divided into a motion block and a static block;then,for the damaged motion block,the outer boundary matching criterion according to the combined texture structure is selected.The relatively optimal motion/disparity vector is subjected to error concealment based on displacement vector compensation,and the reference frame is directly copied for fast error concealment for the damaged still block;finally,the reference frame split and recombination is used to obtain a new motion/disparity compensation block.The quality of the reconstructed block with poor error concealment is improved.The experimental results on different types of multi-view test sequences show that the average peak signal-to-noise ratio(SNR)and structural similarity energy measurement of the depth frame are significantly improved,and the subjective visual quality of the repaired area is closer to the original depth map..2.A whole frame loss error concealment algorithm for 3D-HEVC is proposed by using depth map rendering technique(DIBR).The algorithm is based on 3D-HEVC hierarchical B-frame prediction structure and virtual viewpoint rendering technology and its characteristics.Firstly,the image frame of the viewpoint is received by single view,and the lost image of another viewpoint is drawn.Then,combined with the image structure The fuzzy distortion of the boundary block is repaired based on the time domain information.Finally,in order to enhance the performance,the block motion decision method is used to determine the post processing of the image block,and different types of pixel-based residuals are added to the rendered static block or the motion block to improve performance again.The experimental results show that when we completel DIBR-based error concealment,PSNRof the video could enhance 1.99?6.75dB compared with traditional methods,and SSIM could enhance 0.0119?0.0547.The subjective visual perception quality of the region after the algorithm is improved is better,which avoids the various error concealing disadvantages caused by the difficulty of reconstructing the entire frame image motion vector in the HBP structure.