Research On Error Concealment Of 3D Video For MVD Format

With the replacement of the network era and the development of information technology,people are no longer satisfied with the two-dimensional(2D)visual service provided by the traditional 2D video and gradually turn their attention to the three-dimensional video,which can bring immersive experiences to viewers.The 3D video with MVD(Multi-view Video plus Depth)format can provide viewers with a feeling of on-site presence,which cannot be provided by 2D video and allows users to view stereoscopic video from multiple viewpoints by using multi-view coding technology.Now,The MVD is the mainstream format of 3D video service.Multiview video plus depth video undoubtedly increase a massive amount of data,which must be compressed by MVD coding scheme based on inter-view prediction and inter-frame prediction.However,multi-view coding with high compression degrades the error resistance of MVD video streams.Suppose the MVD video stream has data packet loss during the transmission process,which makes the decode picture error.In that case,the error frame will be propagated within the same viewpoint and propagated across viewpoints.Therefore,how to improve the error resistance of MVD video streams is an essential issue in the field of 3D video research,and error concealment technology is one of the critical technologies for video stream error control.The critical point of the error concealment algorithm for MVD video is to reconstruct the lost area accurately by using the information in the temporal and spatial domain and viewpoint domain at the decoding side.This dissertation focuses on the MVD encoding format in 3D-HEVC.Aiming at the whole texture picture of the MVD video is lost during the transmission process,we proposed two error concealment algorithms based on human visual saliency and virtual viewpoint rendering,respectively.The details as follows:1.We propose an adaptive error concealment algorithm based on visual saliency.The algorithm classified the lost area into low,medium,and high saliency levels by the corrected depth information and texture information in the temporal domain.The temporal and inter-view domain's hybrid search method is used to conceal the lost area for the medium saliency region.For the high saliency level regions,a new motion vector compensation method is designed to solve the low accuracy problem in the previous error concealment algorithm based on HBP(Hierarchical B Prediction).Experimental results on different MVD test sequences show that compared with typical 3D video error concealment algorithms,the average PSNR(Peak Signal to Noise Ratio)of the image reconstructed with the proposed algorithm can be increased by 0.99 d B~2.37 d B,and the average SSIM(Structual Similarity Index Measure)can improve 0.0053~0.0122,and the subjective quality is closer to the original image.2.We propose a 3D video error concealment method using combined with virtual viewpoint rendering technology.Aiming at the inaccuracy of the traditional motion vector extrapolation algorithm under the HBP structure,a new bidirectional motion vector extrapolation algorithm based on DIBR(Depth Image Based Rendering)pixel difference is proposed.At the same time,for the few remaining holes after the motion vector extrapolation,a new hole filling algorithm is designed that extracts the holes from the depth map information and performs partial filling.Experimental results on different MVD test sequences show that compared with the current 3D video error concealment algorithm,the average PSNR of the image reconstructed by this algorithm can be improved by1.22 d B~1.80 d B,and the average SSIM can be improved by 0.0063~0.0091.Compared with the error concealment algorithm based on DIBR,this algorithm can significantly reduce the size and number of holes.