Research On Depth Image Based Rendering And Virtual View Quality Assessment Method

Multi-view video system can provide multiple viewpoints for watching on the decoding side,which brings the viewer a sense of reality and immersion.Due to the bandwidth constraints,it is inadvisable to use dense camera array to capture a huge number of multi-view video data at video acquisition terminal.Therefore,the depth image based rendering technology is utilized.Hence,a small amount of depth and texture video are transmitted and synthesized virtual view video in the decoder.The quality of virtual view video will affect the performance of multi-view video system,and the quality of virtual view video is determined by depth video,color video and synthesized algorithm.Due to the limitation of acquisition equipment and depth map acquisition algorithm,the captured depth video often exits various types of distortion.Depth map pre-processing method can improve the quality of depth video and reduce the distortion of synthesized viewpoint.An effective virtual viewpoint quality assessment algorithm can evaluate the quality of virtual viewpoint and select the better rendering system.This paper intensively studies the effect of depth distortion on the quality of virtual viewpoint,and carries out research on depth video pre-processing,virtual viewpoint quality assessment and virtual viewpoint quality prediction respectively combining the characteristics of human visual perception.(1)The distortion of the depth video will cause distortion in synthesized virtual viewpoint video,while the traditional pixel-based filtering algorithm cannot handle the distorted image block well.Therefore,this paper proposes a depth image processing algorithm based on image segmentation to tackle different depth video distortion.First,aided by the edge and motion information of color image,the image is segmented into many irregular blocks.Secondly,according to the size of the segmentation block,the larger and smaller segmentation blocks are removed.Finally,for each block,the corresponding convergence regions in depth map is statistical analyzed and depth histogram is obtained.The depth relative to the peak in depth histogram is used to fill the distortion region.Experimental results show that the algorithm can effectively solve the problem of depth video block distortion,improve the quality of virtual viewpoint by 0.2dB.(2)The depth image based rendering technology is essential for free viewpoint video systems.However,because of the compression of depth images and limitations of rendering algorithms,various types of distortion might occur in the synthesized virtual viewpoints that cannot effectively be evaluated by traditional two-dimensional assessment methods,thus affecting the visual experience.Hence,a new virtual viewpoint quality assessment method is needed for threedimensional systems.Based on the perspective of human visual characteristics,this paper proposes a method for virtual viewpoint quality assessment using the visual masking effect.First,shift is compensated for in the distorted virtual viewpoint and then the compensated virtual viewpoint is objectively assessed.Next,according to human visual characteristics such as texture,magnitude,and distribution masking,the corresponding visual sensitivity map and visual masks are extracted.Finally,the visual masking and all factors are pooled to create the final quality score.As verified by the experimental results,the method proposed in this paper corresponds with the characteristics of human vision and can serve as a more effective method for assessing the quality of virtual viewpoints.(3)In free viewpoint video system,the quality of virtual view is determined by reconstructed depth and texture video at decoder.In this paper,a virtual view PSNR prediction method is proposed.First,the effect of depth distortion on virtual view quality is analyzed in detail and depth distortion tolerance model(DDTM)which determines the depth distortion range is presented.Next,the DDTM is used to predict the virtual view quality.Finally,the support vector machine(SVM)method is utilized to train and obtain the virtual view quality prediction model.Experimental results show that the Spearman correlation coefficient and root mean squared error of DDTM and SVM method are 0.9501 and 0.4011,0.8865 and 0.7097,respectively.The computational complexity of the SVM method is lower than the DDTM and the state of the art methods.