Research On Image Quality Assessment Based On Perceptual Information

With the booming development of online social media and the popularity of mobile devices,digital images have become an important carrier for people to communicate with each other in daily life.Online service providers generate,store and transmit enormous quantities of visual content every day.To save source and reduce cost,all the images are unavoidable to suffer from a wide variety of distortions during the above process.At the same time,good user experience for those existing social media apps requires high-quality images.Therefore,finding optimal visual quality under the current bandwidth constraints has become a major challenge in the field of multimedia.Since the ultimate receiver of image is human,the purpose of image quality assessment(IQA)is to verify whether the evaluation result is consistent with the subjective feeling of human visual system.Therefore,it is significant to establish objective image quality evaluation methods in line with human visual characteristics.Such methods will benefit the application and development of multimedia greatly.In terms of the human visual system,some visual characteristics and visual factors that affect subjective perception are applied to designing IQA algorithms and achieving satisfactory results.However,there are still some problems in the research field of IQA,such as lack of effective visual perception model,low correlation between extracted image features and image quality,and insufficient consideration of the relationship between visual perceptual features and natural scene statistical features.For stereoscopic IQA,we should focus on the unique quality problems and fully consider the binocular visual characteristics of human eyes.Focusing on the above issue,in this dissertation,based on the sparse representation theory,2D and stereoscopic IQA algorithms have been deeply investigated from the perspective of the visual characteristics and statistical modeling.The detailed contents of the dissertation can be divided into three sections as follows:First,a 2D and a stereoscopic reduced reference(RR)IQA methods are proposed.On one hand,the sparse coding strategy is sufficient to account for the properties of the receptive fields of simple cells in mammalian primary visual cortex,which can be characterized as being spatially localized,oriented and bandpass.On the other hand,the sparse representation has been shown to be highly related to the hierarchical progressive process of human visual perception,that brain processes perceptual information gradually,from primary,structural to more detailed.Therefore,we make use of the characteristic of sparse coding of human vision,a 2D and a stereoscopic RR IQA methods based on sparse representation are proposed,respectively.In the method,we first train a universal dictionary from a set of natural images,and the primitives in the dictionary are regarded as visual perception units.According to the properties of visual primitives in image representation and the visual importance of image reconstruction,all the primitives in the dictionary can be classified into three categories.The distribution statistics of the classified visual primitives extracted by sparse representation is used to measure the visual information,which also refers to Entropy of Classified Primitives(ECP).For stereoscopic image,the ECP of left view image,the ECP of right view image and the mutual information of two view images are the perceptual information.The Maximum(MAX)mechanism is applied to determine the perceptual information under different sparse levels.The perceptual information differences between the original and distorted images are used to predict the image quality by support vector regression(SVR).Experimental results show that the proposed model is effective and highly consistent with subjective evaluation in different IQA databases.Second,a RR IQA metric using sparse representation and natural scene statistics to evaluate the quality of stereoscopic image with asymmetric distortion is proposed.An ideal quality assessment model should simulate the properties of the visual brain to keep consistent with human evaluation.The visual brain appears to have both evolved to seek an efficient,decorrelated representation of image information and to“match” the statistics of the natural image.On one hand,theoretical studies suggest that sparse representation resembles the strategy in the primary visual cortex of the brain for representing natural images.On the other hand,the natural scene statistics have driven the evolution of human visual system and have also inspired the understanding and simulating of visual perception.Inspired by these observations,in this thesis,we propose a novel reduced reference stereoscopic image quality assessment(SIQA)metric using sparse representation and natural scene statistics to simulate the visual perception of the brain.The distribution statistics of the classified visual primitives extracted by sparse representation is used to measure the perceptual information,which is closely related to the hierarchical progressive process of human visual perception.The natural scene statistics of locally normalized luminance coefficients are used to evaluate the natural losses due to the presence of distortions.The perceptual information and the natural scene statistics are used to compute the quality score.Experimental results show that the proposed metric outperforms the state-of-the-art stereoscopic image quality assessment metrics on asymmetric SIQA databases.Third,a series of reduced reference image quality evaluation algorithms with variable reference information are proposed.RR IQA methods make use of partial information or features extracted from the reference image for estimating the quality of distorted images.However,the prediction performance of the algorithm drops as the required reference image information decreases.Tradeoff between the number of RR features and accuracy of the estimated image quality is essential and important in RR IQA.The required reference image information of most RR IQA algorithms can only be fixed,while the bandwidth conditions in real applications often change according to different tasks.Therefore,it is of great significance to design an algorithm with variable reference information and good prediction performance in practical applications.Therefore,a series of RR IQA algorithms are proposed,which differ in the required original feature number.The information required from the reference image ranges from only one single number to the number of overall primitives in the pre-trained universal dictionary.A whole range of methods can be chosen depending on the specific task and the available bandwidth condition,which makes it flexible and useful.The sparse Convolutional Restricted Boltzmann Machines(CRBM)is proposed to train the dictionary,which can provide useful inherent distributed representation of the data.We assume that the primitives in the dictionary are the basic units of visual perception,which contain certain image structure and perceptual information.Different combinations of primitives can represent different levels of structural and perceptual information.Therefore,we can use the multi-level entropy of classified primitives to represent the image quality and to train the quality prediction model.Experimental results show that the proposed model can provide a series of high-performance algorithms with variable reference image information and can solve the problem of adapting the same model to different bandwidths and specific tasks.In summation,a series of new efficient RR IQA models based on the perceptual information are proposed.These models emphasis on the human visual system,using the entropy of classified primitives extracted by sparse representation,fully explore the potential of perceptual information to be used in the field of IQA,significantly improve the consistency between objective and subjective image quality evaluation and the efficiency of prediction,and therefore are of great theoretical and practical importance.