Video Quality Assessment Based On Spatio-Temporal Information

With the fast development of digital communication and multimedia technology,diverse video service applications have obtain more widespread popularization and bring various vision information for human users.In order to provide a satisfactory service quality for human users,assessing video quality accurately becomes a fundamental problem for video service providers.To address this problem,great efforts have been made to design reliable objective video quality assessment(VQA)metrics,which seek to automatically predict the visual quality of videos without the participation of human viewers.Accurate VQA metric can play a central role in different components of a video service system: the benchmarking of processing,compression algorithm and transmission channel settings,performance evaluation of the service system,and finally the selection and optimization of the service system.For the great importance of VQA to video applications,vast number of scientific scholars have conducted in-depth study on this issue.Compared with images,the multi-dimension nature of video signals makes them often suffer from various spatial and temporal artifacts at the same time,and both spatial and temporal information play important roles in human perception as well.Therefore,the key points in VQA task can be summarized as :(1)extracting quality-related features by fully exploring both the spatial and temporal information,in order to measure the degree of spatio-temporal degradation;(2)combining the quality-related features in a proper way to obtain assessment results that are consistent with human's judgements.Focusing on the two key points,a full reference and a no reference VQA metrics are proposed in this paper as follows.1)A full reference VQA metric is proposed in this paper from the aspects of local quality measuring and global spatio-temporal pooling.First,local spatio-temporal volumes are generated around the motion trajectories in the video,and then local quality features are extracted by measureing the spatio-temporal quality degradation within every volume,which can consider the contributions of spatial and motion information simultaneously.Second,a statistical model is utilized to discover the latent relation between the local quality distribution and the global perceived quality,which is more reasonable than other rigid mathematical methods.Experimental results on LIVE database demonstrate promising performance of the proposed metric in comparison with popular full reference VQA metrics.2)Most existing no reference VQA metrics are designed for specific distortion types which are not often aware in practical applications.A further deficiency is that the spatial and temporal information of videos is hardly considered simultaneously.To address these problems,a novel no reference VQA metric based on the spatial-temporal natural video statistics(NVS)in3D discrete cosine transform(3D-DCT)domain is proposed in this paper.Firstly,with the aid of 3D-DCT,both the spatial and temporal statistics of natural videos and distorted videos can be fully explored and analyzed.And then a set of quality-related features are extracted based on the statistical analysis of 3D-DCT coefficients to characterize the spatial-temporal statistics of videos in different views.Finally,these features are used to predict the perceived video quality via the efficient linear regression model.The proposed method is tested on four widely used video databases.Extensive experimental results demonstrate that the proposed method is competitive with the state-of-art no reference VQA metrics and the top-performing full reference VQA and reduced reference VQA metrics.