Video Quality Evaluation Based On Recurrent Neural Network

Entering the 21 st century,with the popularity of short video,live broadcast and other applications,emerging multimedia applications continue to refresh people's previous cognition.Images and videos have profoundly affected and changed our lives.People are exposed to the services of image and video providers all the time,and human life and work are inseparable from digital images and online videos.In some special fields or disciplines,such as medicine,military,survey,security,etc.,image data and videos play a more powerful role than text files,changing the way of working in the past.However,in the daily process of collecting,shooting,transmitting,and encoding video,it is often unavoidable to add distortion to the video data.People cannot accept viewing low-quality video data,but it is time-consuming and laborious to manually judge the video quality.Therefore,it is necessary to design an objective video quality evaluation method for the computer to help us complete this work.In the beginning of the algorithm research on video quality evaluation,researchers chose full-reference and semi-reference quality evaluation,and used manually extracted features to compare with the original undistorted video to predict the video quality score.The non-reference algorithm does not require the initial image or video data to assist in the judgment.More and more researchers are studying the non-reference evaluation algorithm in depth.At the same time,as deep learning shines in the field of computer graphics,deep learning network applications The quality evaluation of non-reference video is also an important issue.The task of video quality evaluation aims to design a model that is simple enough and similar to human subjective evaluation to help humans judge the quality of video.The characteristics of the human visual system are inseparable from human subjective evaluation.Biological and psychological researchers have proposed an attention mechanism in the study of the human visual system,pointing out that when people observe the outside world,they will quickly and automatically focus on the image area of ??interest in the brain.,To help the brain understand the surrounding environment.Therefore,studying the characteristics of the attention mechanism can better help the model to be closer to human subjective evaluation.In order to improve the feature expression ability and make the algorithm closer to human subjective evaluation,the main research content and conclusions of this paper are as follows:1.Fine-tune the convolutional neural network to extract better featuresNow that deep learning is widely used,using deep learning to avoid manually extracting image features is a usable and efficient solution.Therefore,through the transfer learning theory,using the trained VGG-16 convolutional neural network to extract image features can obtain deeper features.This avoids the shortcomings of insufficient training samples in the video set,and at the same time makes the training difficulty of the entire network plummet.From the perspective of the final predictive ability of the model,the features of the convolutional neural network can represent the video quality,and the predictive quality score performs better.2.Improved airspace weight distribution of attention mechanismThe objective evaluation should be close to the subjective evaluation of human beings,and it is indispensable to consider the influence of the characteristics of the human visual system.After exploring the principles of the human visual system and the attention mechanism,the key to the attention mechanism is to calculate the attention score.The idea of calculating the attention score is applied to the model,and the spatial characteristics of the video frame are adjusted to their respective weights.And the prediction results of each feature are fused according to the feature contribution weight.Because the added module parameters are few and can be directly learned during training,there is no difficulty in training the new model,and from the perspective of performance indicators,the effect is higher than the prediction accuracy of the network without modules.3.Recurrent neural network integrating scene switching informationThe distortion or connection between video frames will have a profound impact on the overall quality of the video.If this part of the factor is ignored,the results of the objective evaluation and the results of the subjective evaluation cannot be better linked.Considering the powerful ability of the cyclic neural network to process sequence data,we apply the cyclic neural network to our model to find the time domain characteristics inside the video;considering that scene switching also affects human subjective evaluation,the brightness histogram based on adjacent video frames is adopted.The detection algorithm of the graph is used to obtain the scene switching information.The recurrent neural network model that incorporates the features of scene switching has better consistency with the subjective score of the human eye.This paper deeply studies the attention mechanism and the cyclic neural network that integrates scene switching,and applies them to video quality evaluation.The experimental results show that the spatiotemporal features extracted by the improved attention mechanism and the cyclic neural network model of fusion scene switching can be better Indicates the quality of the video,and has achieved an evaluation result that is closer to the subjective score.