Image Quality Assessment Based On Deep Learning,Spatial And Frequency Domains Analysis

With the high-speed development of multimedia,network and information technologies and the popularity of social software,how to screen,evaluate,restore and enhance image quality from the massive amounts of images has become a hot topic.Objective image quality assessment aims to employ mathematical and computational theory to design an algorithm which correlates well with human subjective evaluation and to predict the quality of distorted image.Based on the perceptual characteristics of human visual system,this thesis mainly investigates the perceptual features of spatial domain,frequency domain and multi-level representations.It primarily researches full-reference image quality assessment method and no-reference image quality assessment method.The main contributions of this thesis include three respects as follows:(1)We propose a full-reference image quality assessment by combining features in spatial and frequency domains.This metric analyzes image quality by multiple complementary features in spatial and frequency domains.Firstly,according to the fact that human visual system pays more attention to the structure region,we extract the gradient magnitude and phase congruency features from spatial and frequency domains,respectively.Afterwards,we analyze the visual impact of spatial frequency and texture information on image quality.Finally,we use the random forest to learn the relationship between various features and subjective perception.Extensive experiments conducted with six publicly available databases demonstrate that the proposed full-reference quality assessment method outperforms all the state-of-the-art full-reference quality assessment methods.(2)Study the no-reference quality assessment method based on deep convolutional neural networks.According to the fact that human visual system has the ability of multi-level representations,we utilize the GoogLeNet to extract the.perceptual features of low middle and high levels to imitate the operation mechanism of primary cortex.Then,we adopt four types of pooling strategies to deal with convolutional feature map of each layer.In addition,we input these features after pooling into random forest for training.The no-reference image quality assessment model can be built after training.We test the proposed no-reference image quality assessment method in four publicly available databases,the experiments prove that the overall performance of proposed method precedes the state-of-the-art no-reference metrics.(3)We propose a novel pooling strategy based on visual weighting in this thesis as traditional max-pooling and average-pooling ignore the weight information between channels in feature maps and do not correlate well with subjective perception.We firstly use the crow-pooling to deal with the channel weighting and spatial weighting in feature map of each layer.Afterwards,we adopt rmac-pooling to process the weighting information in image object region after crow-pooling.Finally,we utilize the traditional global max-pooling and average-pooling to obtain the maximum structural information and smooth information of each layer,respectively.This visual weighting pooling technique owns a simple calculation and combines the global and local pooling strategies to handle perceptual feature map of each layer,it can effectively simulate human visual characteristics.