Blind Image Quality Assessment Based On Classification Guidance And Multi-feature Aggregation

With the rapid development of multimedia technology and the rapid popularization of social network tools,image is widely used as the carrier of information communication and the source of visual information.While image acquisition,transmission and compression may cause distortion with different degrees,resulting in the loss of image information.Therefore,it is particularly important to quantify image quality efficiently and reliably.In most application scenarios,both the reference image and the distortion type information are difficult to obtain.Blind image quality assessment aims to predict the image quality without any reference image information and the distortion type information,so that it has more research value and practical application significance.This project comes from the youth science foundation project “blind image quality evaluation based on deep convolutional neural network”.This paper conducts in-depth research on blind image quality assessment from the perspectives of natural scene statistical features,convolutional neural networks and feature aggregation.The main research contents include the following two parts:The first part is blind image quality assessment in multiple bandpass and redundancy domains(MBRD).First,we extract image features respectively from the bandpass and redundancy domains to capture more sufficient and complementary information,and extract features from multiple color spaces to make the image representation more powerful.Then we stack the extracted features as feature maps,and employ multivariate Gaussian mixture model to describe the image feature maps,then use Fisher Vectors to represent image quality.Finally,a support vector regression model is trained for predicting image quality.Experimental results show that compared with other natural scenes statistics based methods,MBRD achieves better subjective and objective consistency and has good generalization ability.The second part is blind image quality assessment based on classification guidance and feature aggregation(CGFA-CNN).Aiming at the problem of MBRD that artificial design features has limited expression ability and cannot train end-to-end,this paper proposes CGFA-CNN model based on the convolution neural network.First,we construct a large-scale dataset by means of synthesizing distortions to solve the lack of tagged samples,and then pre-trained model guided by classification with the information of distortion levels and types.Then,the responses of convolution layers in different depth is combined to form fused feature maps,and a feature aggregation layer for end-to-end training is designed.Then the final quality aware features can be obtained with the proposed classification-guided gating unit.Finally,the linear regression model is used for mapping the high-dimensional features into the quality scores.Compared with the MBRD model,CGFA-CNN retains its advantage of combining multiple complementary features and adapting to input of any size,and uses convolutional neural network to automatically learn image features,which improves the expression insufficiency of manual features and realizes end-to-end training so that the entire network can be optimized through labeled data.Experimental results show that CGFA-CNN has a great improvement in performance compared with MBRD,and has obvious advantages compared with other methods based on convolutional neural network.