Research On Image Quality Assessment Methods Via Deep Convolutional Network

No-Reference(NR)image quality assessment(IQA)is one of the fundamental area in the field of image processing.The key problem of IQA methods lies in the feature representation related to quality changes in the images.The conventional IQA methods usually design the quality-related features based on the standard datasets with a small number of artificial images.It is not significant to describe the quality changes of the real-world distortions or distorted images in other scenes,resulting in the decrease of the prediction performance of those IQA methods.The research of IQA methods is still challenging.With the accumulation of image data and the rapid progress of computing power,the deep neural network(DNN)in recent years has achieved remarkable results in many applications with its powerful feature learning and representation ability.In this dissertation,we use the prior knowledge of large datasets and the autonomous learning ability of DNN to explore the features related to the changes of image quality,and design the IQA methods for two different scene images of natural images and remote sensing images.The main contents of this dissertation can be divided into four aspects:(1)For the key problem of quality feature representation in the IQA field,the DNN-based image quality feature representation is studied.The autoencoder is first adopted to study the sparse feature representation of distorted images.Then through the transfer learning methods with the pre-trained model of large databases,the deep convolutional neural network(CNN)is trained to explore the feature representation of distorted images.The experimental results show that the extracted deep features can effectively describe the visual distortions of the image with different distortion types and different degrees of distortions.(2)For the quality assessment of single and global distorted natural images,a multi-CNNs-based IQA method is proposed.The traditional methods lack of exploring the characteristics of color information in feature representation.The proposed method combines the deep CNN and the transfer learning to study the quality features related to image colors.Firstly,a group of images are generated with multi-scale transform and multi-color-space transform.Secondly,the general CNN structures are improved to get more useful feature representations.Each image is the input of a single CNN,thus forming a multi-CNNs model.Thirdly,each CNN is trained using transfer learning method with pretrained models.Finally,multiple output feature eigenvectors are fused to construct the quality regression model.The experimental results show that this method has better quality prediction accuracy and generalization performance in several artificial databases and the real-world database.(3)For the local distorted and hybrid distorted natural images,an IQA method is proposed based on deep CNN.The basic idea of the method is a three-step strategy,which is "distortion recognition","specific distortion evaluation" and "comprehensive evaluation",respectively.Firstly,three sets of local distorted images,hybrid distorted images and single distorted images are designed and constructed.Secondly,the classification models for local distortion and hybrid distortion,and specific-distortion quality model are established by training CNN.Thirdly,the predicted specific-distortion quality is weighted by the output probability of CNN,thus achieving the comprehensive quality prediction.The experimental results verify the advantages of the method in the measuring the quality of local distorted and hybrid distorted images.(4)For the visual quality assessment of remote sensing(RS)images,a multi-task CNN-based visual quality evaluation method is proposed.This method measures the RS image quality from two target tasks,which are RS quality grade classification and regression respectively.The first step is to collect RS images from different satellites to construct a RS database with training and testing samples,and a subjective evaluation experiment is performed to get the subjective interpretation ratings for the testing images.Then an automatic calculation algorithm of quality grade is studied to get the subjective quality grades of the training images.A multi-task CNN model is designed for both quality grade classification and regression.The final visual quality is obtained by averaging the predicted label and value.The results on testing images show that predicted quality has a good consistence with subjective interpretation ratings.