Research On No Reference Image Quality Assessment Based On Convolutional Neural Network

No-reference image quality assessment is a fundamental and challenging task in the field of computer vision.Image quality assessment is very challenging due to the lack of reference image information.Precedents such as method taking advantage of natural scene statistical features and method via convolutional neural network(CNN)exhaust all means to extract fragile features related to quality,yet the big breakthrough is still around corner.Targeting the existing problems in the current model of image quality assessment based on CNN,this thesis proposes corresponding solutions.The main contributions are as follows:(1)An image quality assessment method based on multi-scale CNN is proposed.Targeting the problem that only high-level features which are insufficient to describe the changes of image quality are used in image quality assessment model based on CNN,this algorithm proposes to use four branches with different convolutional layers which possess different receptive fields to form a multi-scale network to extract features which are able to reflect changes of image quality.The network not only preserves the highlevel features under the coarse scale which are consistent with advanced perception of human brain,but also preserves features which are more able to reflect the quality changes under the fine scale,so that the network's perception process of image quality is more similar to the human subjective perception of image quality.Besides,in order to make up for the shortage of training data,this algorithm introduces data enhancement methods such as random change of image contrast,saturation and hue into the field of image quality assessment for the first time,which satisfies the requirements of the deep network for training data and improves the accuracy of the model.Experiments show that the quality score predicted by this model is highly consistent with human subjective perception.(2)An image quality assessment method based on CNN with multiple attention mechanism is proposed.Aiming at the problem that current model is not capable of selecting effective features adaptively,this algorithm introduces multiple attention mechanism into CNN to optimize the image quality assessment method.The network is focused on learning effective features that can describe changes in image quality via attention mechanism.The feature modulation mechanism enables the network to selfadaptively select features that effectively describe image quality from shallow convolution layers and fuse them with features of deep layers;channel selection mechanism enables the network to self-adaptively avoid the interference of invalid channels;non-local mechanism expands the field of convolution layer from local area to non-local area,measuring the relevance of the pixels of the current position and remote position,so as to measure the effect of local distortion on global quality in nonlocal receptive field.The three attention mechanisms improve the perception process of image quality from different levels,making the network concentrate on learning the features that effectively represent image quality,so the model is able to avoid the interference of invalid features,and improve the consistency between the prediction effect of the network and human subjective perception.The experimental results show that the model has achieved excellent results on many databases.(3)A no-reference image quality assessment method is proposed to enhance the interpretability of the network.Current image quality assessment models view CNN as a black box,which means the model can not explain the basis of the network prediction,or the specific role of each convolution kernel in predicting the image quality of different types of distortion.In order to overcome this shortcoming,this algorithm combines U-shaped neural network and CNN to extract features and predict scores as a whole,and the output of the U-shaped network is used as the middle layer feature map of the model,and the mean square loss function is used to constrain the feature map approximate the structural similarity map which is equivalent to the introduce the middle layer feature map supervision for the model,so that the middle layer feature map of the model reflects the changes of image quality from the three attributes of the image: brightness,contrast and structure,thus providing the basis of the quality score obtained by the model.This algorithm also adds loss functions of distortion classification to each convolution kernel in the deep convolutional layer,which is equivalent to introduce distortion class supervision for the convolution kernel,constraining each convolution kernel to learn the feature representations of different kinds of distortion,thus making each convolution kernel work for distortion of specific type,and making it easier for humans to understand the specific role of each convolution kernel plays in predicting the quality of images with distortion of different types.While maintaining the consistency with human subjective perception,the model also enhances the interpretability of the middle layer feature map and the working mechanisms of convolution kernels.