Video Quality Assessment Based On Deep Networks

With the rapid development of multimedia technology and internet technology,images/videos are widely used as the carriers of information,which more intuitively show things in the real world.Before the final presentation to the user,the images need to undergo several processing stages such as acquisition,compression,storage and transmission,during which time distortion can be caused easily,which results in a degraded image/video quality.Therefore,the research on image/video quality evalua-tion method is more and more important.With the rapid development of deep learning,convolutional neural network has made a breakthrough in computer vision.However,most of the existing models of image quality assessment based on convolutional neural networks still only include one convolutional layer.As a powerful feature learner,the deep convolutional neural network can learn the high-level semantics features directly from the image,and the deep convolutional neural network shows strong performance in computer vision tasks such as image classification and target detection.Therefore,in the light of the shortcomings of the existing methods,this paper proposes a series of non-reference image/video quality evaluation methods:(1)This paper presents a non-reference image quality assessment method based on multi-task AlexNet.First,for different distorted images may have different sizes,the local contrast normalization is applied and the non-overlapping 32 x 32 patches are cropped from the distorted images.Then,the multi-task strategy is employed to integrate the distortion type recognition and image quality assessment task into the convolutional neural network.The parallel multi-task strategy can make the network learn more robust features.In the test phase,the distortion type and the quality of the image patches can be evaluated by the multi-task AlexNet and then the selection weight method is utilized to assess the quality of the raw image.In experimental part,the effectiveness of the algorithm is verified in the LIVE image quality evaluation database.(2)This paper presents a non-reference video quality assessment method based on multi-task AlexNet.Due to the superior performance of the non-reference image quali-ty assessment model based on multi-task AlexNet,this paper extends the non-reference image quality assessment model to the non-reference video quality assessment task.First,we fine-tune the pre-trained convolutional neural network for image quality as-sessment on the video data to predict the quality scores of the video frames.Then,the temporal hysteresis model is employed to amend the quality scores per video frame.Finally,the mean of amended video frame quality scores is taken as the quality scores of raw video.The performance of the proposed model is verified in the experimental part.(3)A novel video quality assessment method based on deep convolutional neu-ral network is proposed to assess the video quality blindly.This method is derived from further extending multi-task AlexNet to video quality assessment task.First,the pre-trained convolutional neural network is fine-tuned on the LIVE video database to transfer the convolutional neural network for image quality assessment to the video quality assessment task.Then,the network is utilized as the feature extractor to extract the feature of each video frame patch.Next,the vector of locally aggregated descriptors is employed to generate the representation of each video,and the dimension of video representation is reduced by principal component analysis.Finally,the support vector regression is used to get the quality score of each video.In the experimental section,this model shows superior performance.