Video Quality Assessment Based On Video Content Perception

Video is the most intuitive and vivid information carrier.More and more videos apply in all walks of life that bring convenience and interest to people's work and life.Video quality can interpret as the ability of correct information transmitting and the degree of required satisfaction.It is essential for video quality to transmit information correctly and use video conveniently.However,distortion is introduced to video on account of a series of operations of video production,compression,transmission,leading to quality degradation.Thus,it is necessary to access the quality of video,ensuring an excellent viewing experience and video service for users.Video quality assessment(VQA)has become a hot research because of its important role in visual experience.This thesis carries out systematic study and in-depth research on video quality assessment based on the process of visual perception.The main research in this thesis are as follows:(1)A video quality assessment based on spatiotemporal masking effect is proposed.The objective video quality assessment predicts the quality of video by counting all of distortions on the video.But the local distortions are ignored by human because of the visual masking effect,the objective assessment has a bad consistency with the subjective score.A spatiotemporal masking model is constructed to make the objective assessment consistent with the manner of human perceive distortion.The spatial masking coefficient is computed by the spatial randomness for the spatial masking effect,and the temporal masking coefficient is calculated by the eccentricity,magnitude of motion vectors and coherency of object motion for the temporal masking effect.The spatial and temporal masking coefficient are fused by a non-linear combination to quantize the visual masking effect.Since the impact of the frame with a huge change of quality on the overall assessment is significant,a score difference pooling strategy is proposed to replace the average pooling.The frame with a significant difference in quality from the previous frame or severe distortion is chosen to be used in the temporal pooling stage.Finally,the structural feature is extracted to predict video quality.Experimental results demonstrate the proposed algorithm models the perceptual process of video quality effectively and has a good consistency with subjective perception.(2)A blind video quality assessment based on pseudo-reference is proposed.For the problem of no-reference algorithm has difficulty in extracting features that can reflect the change of video quality effectively,the characteristics of the previous frame is used as pseudo-reference information of the current frame,and the pseudo-reference information is compared with the current frame by a full-reference algorithm to extract the pseudoreference feature.Since different scene transformation and motion in video can cause the deviation in the human eye perception of the same distortion,the dynamic feature extracted from the dynamic change of video scene computed by Gram matrix coefficient is utilized to describe the degree of deviation.Last,the pseudo-reference and dynamic features are used to predict video quality by the support vector regression.Experimental results indicate the performance of the proposed algorithm is better than others.(3)A no-reference video quality assessment based on inflated three dimensional(3D)convolutional neural network(CNN)is proposed.VQA based on the two dimensional(2D)CNN evaluates the degree of video quality descent in spatial and temporal domain respectively.But the change of video quality in spatial and temporal domain is correlative,VQA based on 3D CNN can assess the overall change of video quality.The inflated 3D convolution is obtained by repeating the weight parameter of 2D convolutional kernel in the third domain.The multiscale spatiotemporal features which can reflect the degree of video quality descent is extracted by the inflated 3D CNN consists of the combination of inflated 3D convolution and Inception.And non-local attention is added to make the network concentrates on areas that have a huge impact on video quality.Thus,the assessment model conforms to visual perception characteristics.Most of existing VQA databases are small.To extend the dataset,image quality assessment database is transferred in training set by copying the distorted image along the temporal axis to generate “pseudo-video”.Experimental results demonstrate the proposed algorithm has a good consistency with subjective perception.