Image Quality Assessment Methods Based On Visual Perceptual Characteristics

The fundamental purpose of image quality assessment(IQA)is to automatically evaluate the image quality by computer algorithms.In order to make the evaluation results fit human subjective perceptions,designing image features that conform to human visual perceptual characteristics has become the critical part in the research of IQA methods.Limited by the complexity of human visual system(HVS),there are still no image features that can accurately describe the entire human visual perceptual characteristics.However,with the development of the HVS in the fields of visual physiology and visual psychology,a variety of research results on human visual perceptual characteristics promote the continuous improvement of IQA methods.The applications of HVS characteristics in the IQA field are mainly studied in this thesis.Specifically,based on the existing research results of human visual perceptual characteristics,three types of images: natural scene image(NSI),synthesized view image(SVI)and screen content image(SCI)are involved in this thesis,and four IQA methods are proposed for different application scenarios.(1)A full-reference(FR)NSI quality assessment method is proposed based on the sparse structural similarity.Firstly,the local sparse structure of images is defined by using the structural characteristic of sparse representation coefficients,which can simulate the responses of simple cells in the visual cortex to visual stimuli.Then,the sparse structural similarity metric is defined by comparing the change of local sparse structure between the distorted and the reference images.Meanwhile,in order to compensate for the lack that the image luminance can't be well described by sparse structure,the local quality of images is proposed to be measured by combining the local luminance similarity with the sparse structural similarity.In the pooling stage,the regions with higher contrast are selected to calculate the overall quality scores of images according to the human visual saliency.Experimental results show that the quality of NSI can be effectively described by the proposed quality-aware feature,and the evaluation results of the proposed method are highly consistent with human subjective perception.(2)A reduced-reference(RR)SVI quality assessment method is derived based on the feature covariance matrix.Firstly,a variety of low-level visual features are extracted to simulate the multi-faceted visual perceptions of image quality.Then,the covariance matrix is employed to simulate the information fusion mechanism of the human brain,and all low-level visual features are merged together as the overall perception of image quality to the human brain.Finally,the SVI quality is measured by calculating the distance of feature covariance matrix between the synthesized view image and the reference image.Experimental results prove that the feature covariance matrix can effectively describe the quality of SVI.Moreover,the low-level visual features can be flexibly selected in the proposed method according to application requirements and communication conditions,and thus,the system resources can be fully utilized by the proposed method.(3)An opinion-aware no-reference(NR)NSI quality assessment method is developed based on the Difference of Gaussian(Do G)natural scene statistical(NSS)feature.Considering to the center-surround characteristics of receptive field and the multi-frequency channel perception of the human retina cone in the HVS,the NSI is decomposed into several Do G sub-bands by Do G model.Then,the discriminability of the joint feature can be enhanced by taking the advantage of the feature extracted on each Do G sub-band.Finally,the machine learning framework is leveraged to learn the regression model between the quality-aware features and human subjective opinion scores,and the regression model can be used to predict the quality of distorted images subsequently.Experimental results demonstrate that the proposed method can effectively improve the discriminability of the image quality-aware feature and promote the evaluation accuracy of NR-IQA method for multiple types of distorted images.(4)An opinion-unaware NR SCI quality assessment method is designed based on the visual internal generative mechanism(IGM)of the human brain.The algorithm is proposed by combining local structure feature with global luminance and texture features to comprehensively describe the quality of SCIs.In term of the local feature,the local most preferred structural(MPS)feature is proposed based on the IGM of the human brain.In term of the global feature,the joint luminance statistical features and the joint local binary patterns(LBP)statistical features are employed to describe the global luminance and texture characteristics of SCIs.In addition,in order to get rid of the constraint of human subjective opinion scores,a large-scale training dataset with 80,000 distorted images is constructed,and a high-performance FR SCI quality assessment method is employed to automatically generate the objective quality labels of those training images.Finally,the machine leaning framework is leveraged to learn the regression model between the quality-aware features and objective quality labels from the generated large-scale training dataset,and the quality of distorted images can be predicted by using the regression model.Experimental results prove that the proposed method can obtain consistent results with human perception without any subjective opinion scores.