Study On Consistency Enhancement Technique For Depth Video

With the 3D application in the popularization of the entertainment industry,manufacturing industries,such as 3D TV,free viewpoint TV,virtual reality,3D games and robot vision,etc.variety of 3D application has quietly come to our life.Multi-view video system is based on DIBR technique,and it is well known that to obtain high quality and accurate depth information of the real scene is the one core problem in computer vision field.To obtain each frame of depth map is operated separately by depth estimation algorithm,and the estimation accuracy is limited,at the same time,it also weaken inter-view consistency of depth videos.The consistency in time domain of depth video leads to artifact and flicker emerging in synthesis' s view,which poor observers' experience.The inconsistency had significant effect on the quality of synthesis virtual view,and led to the decline of multi-view coding performance.In view of temporal consistency enhancement,most of the existing algorithms can not enhance the consistency for a long range in time domain,and the strength of effect is weak.Compared with the time domain,the consistency enhancement of inter-view depth information is more challenging because of the different perspective,occlusion region,the influence of lights and shadow existing in another view.Namely the above problems,this graduation dissertation puts forward the following consistency enhancement methods for multi-view depth videos:(1)Aiming at the drawback of the temporal coherence enhancement method in depth estimation and the inaccurate refinement on depth map,the weak strength of improvement,the short range of effect lasting in time domain among the most methods.This paper proposes a method to enhance the temporal consistency of depth video based on the motion intensity of scene,inconsistent depth pixels are refined for a long range in time domain with the relative more consistent depth value selected by motion intensity measure and motion region segmentation,and the weighted mode temporal filtering is applied on refined depth video to enhance the spatial continuity and temporal consistency of depth value to generate well optimized depth video further.The result of experiments shows that this proposed method do enhance the temporal consistency of depth video,and save encoding bit rate ranging from 17.48% to 31.75%,while it improves the subjective quality of rendered virtual views.(2)To solve quality decreasing of synthesis virtual view caused by inter-view inconsistency of depth videos,this thesis proposes an adaptive inter-view consistency enhancement for multi-view videos.Firstly,consistency checking is performed between left and right views,meanwhile,the consistency evaluating value is created,and it divides both views into 3 kinds of regions,including occlusion region,consistent region and inconsistent region.Combined with the edge information of the depth map and block tag after depth clustering segmentation,adaptive joint bilateral filter is designed for consistency enhancement.Experiment results show that the PNSR of rendered image boosts 0.13 dB by an average,preprocessing improves both objective and subjective quality of synthesis virtual view.(3)To improve the effect of multi-view depth videos compression,this thesis put forward a novel consistency enhancement for multi-view depth videos.This algorithm is applied based on three neighbor views of depth videos.There is a depth information exchanging framework designed for inter-view depth videos,then blend the depth value among the three depth videos with the framework and consistency checking to get a new one.Later,SIFT algorithm is performed to find Scale-invariant features and make judgment for affine characteristics to determine the matched local region.The local and global consistency among neighbor depth videos are enhanced by weight mode optimized blending and iterative processing based on consistency evaluating value.Results of experiment show that this preprocessing for multi-view depth videos can save bit rate ranging from 4.10% to 13.68%.