No Reference Video Quality Assessment Based On Deep Learning

Nowadays,due to the popularity of smart devices and the rapid development of mobile networks,we are surrounded by various surveillance and live video.People can not only enrich their entertainment life by watching videos,but also get a lot of knowledge from videos,so video plays an increasingly important role in people's lives.However,video is often prone to distortion in the process of recording,compression coding,transmission,etc.The quality of video directly affects people's subjective feelings about video.When judging video quality as an artificial work,it is time-consuming,laborious and subjectively affected.Therefore,it is necessary to evaluate the video quality objectively by computer.In objective video quality assessment,most researchers use traditional manual feature extraction and shallow learning machine to predict video quality score,so the manual extraction of features in stages makes the results less than ideal.With the development of deep learning,more and more researchers have begun to use deep learning to solve image quality assessment.Since the video quality assessment has more difficulties than the image quality assessment,it is necessary to consider the timing of the video and the small sample at the same time,which leads to relatively little research in this area.However,because the convolutional network has excellent performance in feature extraction,this paper overcomes the above problems and uses the deep learning method to study the objective quality assessment of video.The contents and results of the research in this paper are summarized as follows:(1)A blind reference video quality assessment method based on time-space domain feature extraction is proposed.The method considers the spatio-temporal characteristics of video frames.Firstly,the frame difference map is used to extract the time-domain features of video quality degradation caused by jitter.The space-time domain feature of the distorted video is extracted by the convolutional neural network on the frame difference graph,and then the feature is further reduced by using PCA.Finally,using Xgboost to fit features linearly to get the final objective quality score of video.Experiments in the existing video quality assessment database show that the method in this chapter can predict the quality of video well.(2)A blind reference video quality assessment method based on migration learning is proposed.The method avoids the cumbersome assessment of phased video quality and adopts an end-to-end network structure for video quality assessment.In the experiment,the end-to-end convolutional neural network is first constructed,and the parameters of the VGG-16 convolutional layer are migrated to achieve the purpose of speeding up convergence and shortening the training time.Then the video frames are cut and stitched.Finally,the stitched samples are sent to the network for end-to-end fitting.The method of this chapter can avoid finding features in stages,and then carry out regression prediction to effectively improve the accuracy of video quality assessment.The experimental data of the existing video quality library shows that it is very good to achieve good subjective consistency with the human eye.(3)A blind reference video quality assessment method based on recurrent neural network is proposed.This method is a good solution to the extraction of temporal and spatial features of video.This experiment is based on the improvement of method(2),migrating the parameters of VGG-16 convolution layer,and then constructing a cyclic neural network,that is,convolution network is used to extract spatial information,and cyclic neural network is used to extract temporal information.The combination of the two,in the video quality assessment library,can reflect the subjective score of the video well,and the method of this chapter is also end-to-end training and prediction,which improves the prediction accuracy.