Error Concealment For Three-dimensional Video System

The applications of three-dimensional(3D)video have drawn increasing attention as a new multi-media technique for providing high-quality,more wide-angle vision and immersive experience to end users.In order to represent a 3D scene,multi-view video plus depth(MVD)format,which can provide the depth geometry information and the video information from different angles of the stereo scene,has emerged as an efficient data representation for 3D video.However,the presence of multiple cameras as well as corresponding depth information significantly increases the amount of data.The compressed video data with high coding efficiency is more sensitive to transmission errors shuch as data-packet loss or entire frame loss.Therefore,error concealment for frame loss in 3D video is a very important research topic.For these reasons,this thesis mainly focuses on exploiting the several correlations of the multi-view sequences,and considering the content of pictures and the characteristics of the HVS(Human Visual System)to design efficient error concealment(EC)techniques for 3D video transmission.In order to further study on the quality assessment of the reconstructed 3D images affected by packet loss or bit dislocation in the decoding end,the algorithm about local and global sparse representation for no-reference quality assessment of stereoscopic images is proposed.In summary,the main tasks of this thesis are listed as follows:(1)In this paper,for the entire frame loss in the MVD_HBP structure,new error concealment is proposed by using adaptive selection of multiple prediction correlations.At first,a model about adaptive selection of multiple prediction correlations is established according to comprehensive analysis of the content characteristics in the lost frame conducted by applying structural similarity(SSIM)index metric.Then,temporal/inter-view prediction regions of the lost frame are estimated.Finally,we explore bilateral temporal/inter-view error concealments which are applied in the temporal/inter-view prediction regions.Experimental results show that the proposed algorithm provide a better image quality than conventional methods both objectively and subjectively.(2)In the second part,a scalable EC algorithm for the whole B-frame loss in 3D video transmission is proposed.First,basing on the view type of the lost fame and the different reference layers which the lost fame is in,our scheme divide the lost frames into two types: 1)key frame;2)ordinary frame.And then,the proposed algorithm divide the pictures into motional or static estimation regions by utilizing the reference frames.Different approaches are used to conceal different lost frames in different regions.It improves that the whole quality of the restored picture and the proposed algorithm effectively reduces the complexity of the algorithm.The subjective and objective quality of error concealment images is improved in the multi-view video.(3)In this paper,a bilateral error concealment algorithm of the depth entire frame loss for 3DTV transmission is proposed,which utilize the strong temporal correlations and moving correlations between the depth video and its corresponding 2D video.Experimental results show that the proposed algorithm could improve the subjective and objective quality of the reconstructed image.(4)We propose an error concealment algorithm based on the depth maximum tolerated geometry distortions(DMTGD)for the depth frame loss.First,the distribution figure of depth maximum tolerated geometry distortions is derived according to the relationship between the depth map distortion and the corresponding just-noticeable difference(JND).The JND is based on the colour map corresponding to the lost depth map.The algorithm adjusts the error concealment strategy according to the value of DMTGD in the different regions of the depth map.Simple concealment is used in the regions with the larger value of the DMTGD to improve the efficiency.Otherwise,complex concealment is used to improve rendering quality of the virtual viewpoints.(5)In this paper,we propose a blind quality evaluator for stereoscopic 3D images by learning local and global sparse representations.Specifically,at the training stage,we first constructe a large-scale training set by simulating some common distortions that are likely encountered by stereoscopic images,and propose a multi-modal sparse representation framework to characterize the relationship between the feature and quality spaces for all sources of information from left,right and cyclopean views in local and global manners.At the testing stage,basing on the derived 3D quality prediction framework,the local and global quality scores from different sources are predicted and combined to drive a final 3D quality score.Experimental results on three 3D image quality databases showe that,in comparison with the existing methods,the proposed algorithm can achieve better prediction performance to be in line with subjective assessment.