Reserch On Depth Distortion Models In 3D Video Coding

With the development in technology of computer and communication,3DV(Three Dimension Video)instead of 2DV(Two Dimension Video)will be the next generation mainstream video applications gradually.3DV could provide a natural and live visual experience in immersion.Auto-stereo displays make the audience not only never be bound by glasses,but also freely choose the viewpoint within a certain range.And a 3DV system presents 3DV of the viewpoint chosen by the audience.The huge amount of original multi-view videos becomes the challenge for information transmission infrastructure.Restricted to the limited transmission bandwidth,we cannot transmit 3DVs of all viewpoints.The depth enhanced data format includes only colour videos and depth data of the referred viewpoints.Depth data control virtual view synthesis.And the compression distortion of the depth video declines the quality of synthetic virtual view.So,it is of importance to study the mechanism of depth video distortions on synthetic virtual view quality.On the one hand,the Qo E(Quality of Experience)will be ameliorated with the help of the improved quality of virtual view if the distortions of compressed depth video are effectively controlled in 3DV coding.On the other hand,3DV coding performance can be improved by suppressing the depth distortions under JNDD(Just Noticeable Depth Difference)threshold because not all the depth distortions are be perceive.In this paper,3DV coding distortion mechanism is studied.The major work of this paper is as follows:At first,the influence of the depth of distortion on virtual visual distortion is studied.On this basis,a depth based virtual visual distortion model is built.First,depth maps are divided into flat units and non-flat units.The virtual distortions in a flat piece are calculated as a whole using frequency domain method.The occlusion relationship may be altered by the depth distortion in edge areas.As a result,per-pixel distortion calculation as well as occlusion relationship analysis is performed in non-flat pieces.We not only calculated the distortions induced by the mistakenly occluded pixels,but also the distortions from the mistakenly un-occluded pixels.Although the retio of the edge area in a depth map is small,virtual view distortions from the edge area are significant.In order to accurately classify the units,we adopt a disparity based classification criterion.The disparity based classification criterion is a function of scenes parameters and shooting parameters so that the classification threshold can be adjusted.The average mean square error difference((35)MSE)between the estimated distortions and the measured distortions is ruduced to 2.9 in the proposed model.Then,we build a MJNDD(modified JNDD)model,a JNDi D(Just Noticeable Disparity Difference Model)model and a JNPDD(Just noticeable perceived depth difference)model based on HVS(Human Visual System)physiological mechanism of stereo vision.MJNDD model is built using a three-piecewise linear function.The Linear Correlation PLCC(Pearson Linear Correlation Coefficient)of the predicted threshold of the MJNDD model and the data from subjective experiment reaches0.99,which is high than the two-piecewise linear model and the four-piecewise constant model.We assume that eyes always gaze at the virtual targe and maintain the same convergence state for a virtual target and its real target positioned in the JNDi D model,which provides the basis for express the the JNDi D thresholds in various stimulated depth spaces in one function family.JNPDD model adopts a function family united by JNDD thresholds in natural scenarios.The JNPDD model integrates displaying and viewing parameters so as to be applied cross different displays.Finally,the depth based virtual view distortion model and JNDi D model are applied in 3DV coding.We put forward a perceptual 3DV coding method oriented to the virtual view distortion,in which rate distortion criterion is optimized based on the virtual view distortion model and predicted residuals are filtered according toJNDi D thresholds.The experimental results show that the proposed algorithm reduces the bit rates and remains the visual perception quality.This further confirms that the proposed depth-based virtual vision distortion model and JNDi D model are effective in 3D coding applications.