| Free Viewpoint Video(FVV)gets the further development on the basis of threedimensional video,which enables users to experience stereoscopic perception from different angles and increases their immersion in watching video.Depth Image Based Rendering(DIBR)technology plays an important role in FVV system.A small amount of reference viewpoint information is transmitted at the encoder side,and multiple virtual viewpoints can be obtained at the decoder side by DIBR technology,so as to achieve the purpose of reducing the amount of transmitted data.However,the immature DIBR technology result in many distortions in the rendered virtual viewpoint video images,which seriously affect the user's viewing experience.When users watch stereo video in FVV system,there are many combinations of left and right viewpoints.Different combinations bring different visual perception to users.Exploring the objective quality of virtual viewpoint stereo video can effectively guide video encoding and optimize the rendering process as feedback,thus improving the FVV system.The virtual viewpoint video is composed of multiple frames of synthetic viewpoint images.The research of quality assessment of synthetic viewpoint stereo image is the basis of research of quality assessment of virtual viewpoint stereo video.Therefore,in this paper,the quality assessment of synthetic viewpoint stereo image is studied firstly;then,the quality assessment of virtual viewpoint stereo video is depth studied from the perspective of human visual characteristics.(1)Aiming at the quality assessment problem of synthetic view stereo image,a multi-feature fusion method is proposed.Considering the specialty of the distortion in synthetic image and the feature of binocular perception,the features of the distortion areas,the distortion edges and the singular value,are fused.Firstly,the distortion areas corresponding to the human eyes are obtained by using the parallax and the threshold,and then the mean structural similarity(MSSIM)in these areas is calculated.Secondly,the distorted edges in the synthetic viewpoint edge image are extracted,and then the MSSIM value is calculated.Thirdly,the difference between the original image and the distorted image singular value is calculated.Finally,the three features are fused to obtain the final objective quality score.The method of this paper and the comparison methods are tested on the synthetic viewpoint stereo image library.The experimental results show that the Pearson linear correlation coefficient(PLCC)and the Spearman rank order correlation coefficient(SROCC)of this algorithm are 0.865 and 0.862,respectively.Compared with the existing assessment methods,it can better reflect the quality of synthetic viewpoint stereo images.(2)Aiming at the asymmetrical distribution of left and right viewpoints of virtual viewpoint stereo video,a no reference quality assessment method of virtual viewpoint stereo video is proposed.Considering the characteristics of asymmetric distortion of the virtual viewpoint stereo video and the user's human visual characteristics,this method uses local and global features to represent the quality of virtual viewpoint stereo video.Firstly,the temporal and spatial morphological edges of video in three scales are extracted,and the horizontal and vertical standard deviation features are calculated.Secondly,the temporal and spatial natural scene statistics features of video in three scales are extracted.Finally,considering the asymmetrical distortion distribution of the left and right viewpoints,the features of the left and right viewpoints are combined into the stereoscopic viewpoint features according to the amount of information reflected in the shearlet subbands.The final objective quality score is obtained by random forest prediction.The method of this paper and the comparison methods are tested on an asymmetric virtual viewpoint stereo video database.PLCC,SROCC,Kendall rank order correlation coefficient,and root mean square error of this algorithm are 0.908,0.867,0.699,and 5.806,respectively.Compared with the existing quality assessment methods,the results of this method are more consistent with subjective perception of the human eye. |
PDF Full Text Request