| With the rapid development of mobile internet,digital video applications such as Zoom,We Chat,Tik Tok and Kwai have become important components in people’s remote work,learning and entertainment processes through the widespread use of smart devices.How to accurately measure users’ actual perceived experience of video has become the key to improving video quality and promoting the development of the digital video industry.Human beings as evaluators of video quality,predicting the video quality scores by simulating the Human Visual System(HVS)perception mechanism is an effective way to solve the Video Quality Assessment(VQA)problem.However,traditional Full-Reference VQA(FR-VQA)or most current deep learning based FR-VQA methods generally adopt a "data-driven" approach,requiring tedious feature processing,including video pre-processing,feature extraction and dimensionality reduction,which are not closely related to the actual human visual system perception process.Therefore,this paper takes the multi-scale spatio-temporal joint perceptual properties into account,combines the Reverse Hierarchy Theory(RHT)in the HVS,simulates the human visual perception process of distortion sequences at different spatiotemporal scales,and conducts research on FR-VQA methods under real scenes.The main contents include the following two aspects:(1)The thesis proposed a FR-VQA model with multi-scale spatio-temporal feature aggregation.The spatio-temporal scale is an important attribute of video,and single granularity features extracted only at a fixed scale are not sufficient for global distortion representation.In order to extract and aggregate rich multiscale feature to portray the human complex perception mechanism,this paper firstly combines the image structure distortion and perceptual motion energy to adaptively sample the sequence,solving the pain point of losing key frames caused by fixed interval sampling in traditional VQA algorithms.Secondly,to extract different granularity features to characterize video content distortion,Long Short Term Memory(LSTM)layers of different lengths are introduced to extract multiscale spatiotemporal semantic features of the sequence.Thirdly,to explore effective ways to aggregate multiscale features,the perception feature fusion of LSTM layers of different lengths is simulated by RHT’s positive and reverse perception iteration mechanism.Finally,a multi-channel self-attention module is used to regress and predict the score.This model achieves optimal or nearoptimal performance with SRCC above 0.93 in multiple databases.(2)The thesis proposed a FR-VQA model based on the RHT.The first part of the work is further deepened by exploring the differences between the human perceptual mechanisms of the feedforward and feedback progressive processes based on RHT,and memory units suitable for forward and reverse perception are designed,respectively.Firstly,this paper proposes a multi-scale spatial attention redistribution module to extract frame-level spatial features frame by frame.Secondly,the Forward Perception Module based on Eureka effect(FPM)is designed to describe the strong and persistent positive perception,taking into account the Eureka effect in cognitive learning and focusing on the influence of the first perceptual experience.In addition,a Backward Perception Module based on Short-term dependencies(BPM)was introduced to describe mild and transient backward perceptions.Finally,to allow sufficient interaction of features between different layers,the multi-layer features are aggregated with stacked FPM and BPM layers to predict the final video quality.The experimental results show that the performance of this algorithm outperforms the current mainstream algorithms,and the ablation experiments demonstrate the effectiveness of the FPM and BPM modules. |
