Spatial-temporal Fusion-based Video Quality Assessment

With the development of multimedia and network transmission technology,there are increasing demands on the high visual quality of short videos and live streams.Video distortion in spatial and temporal domain may be introduced in shooting,storage,encoding,transmission and displaying,which leads to the degradation of visual quality and affects the watching experience of users.In order to effectively evalute visual quality of video,video quality assessment is becoming more and more important in the field of image and video processing.The traditional image quality assessment models often focus on evaluting the spatial distortion of video,but cannot evalute the temporal distortion,which is not consistent with the subjective perception of human eyes.In addition,intra and inter encoding information in video streams are related to the spatial and temporal quality of video.However,many video quality assessment models are usually applied to the decoded videos,which ignores the encoding information of videos.This thesis studies the difference of spatial-temporal features between reference videos and distortions videos and the relationship between encoding information and distortion,and then proposes spatial-temporal fusion-based video quality assessment methods.The main research contents are as follows:(1)We obtain the difference of gradient amplitude between reference videos and distortion videos and use it to evalute spatial quality.Besides,considering human eyes have diverse sensitivities of the ascend and descend of distortion in adjacent frames,the gradient of spatial quality is calculated and as the descriptor of video fluctuation after percentage pooling.For temporal quality,motion vectors of adjacent frames are calculated by motion estimation algorithm to describe temporal information of video.The changes of local statistics of motion vectors between reference videos and distortion videos are used to evalute temporal quality.Finally,a temporal-spatio fusion based full reference video quality assessment model is constructed,the feasibility and validity of the model is verified on three video quality assessment datasets.(2)Intra and inter encoding information in video streams are highly related to the visual quality,such as coding tree partition depth which is related to texture complexity,quantization parameter value which can represent the degradation,and motion vectors which can represent movement strength.In order to study the correlation between encoding information and visual quality of videos,a new video stream quality assessment dataset is constructed which encoded by HEVC.By fusion of the encoding information which extracted in decoding side,a encoding information-based video quality assessment model is proposed.In this model,the gradient boost regression tree is adopted to fuse features.We verify the effectiveness of our proposed method based on our constructed dataset and the experimental results demonstrate its good performance.