Image Quality Assessment Based On Image Content

With the growing popularity of mobile devices, mass storage devices and Internet services, people can easily generate, store and transfer images, videos and other mul-timedia data. The explosive growth of image data not only brings us abundant useful information resources, but also brings to image processing algorithms and computer vision applications a greater challenge. Acquisition, encoding, storage, transmission and display process of images may cause deterioration of the image quality, thus affect-ing human perception or the performance of computer vision algorithms. Image visual quality assessment is a fundamental research of computer vision. Most current research focus on the objective image quality assessment methods which does not require hu-man intervention. These algorithms are based on low-level or statistical characteristics extracted from spatial or transform domain like DCT, Wavelet, etc. without usage of high-level semantic information of the image.To address this shortcoming, this paper presents a new non-distortion-specific non-reference image quality assessment algorithm based on both high-level semantic infor-mation and low-level characteristics of image. The main work and innovations of this paper is as follows:(1) We research and conduct experiments to explore the relationship between im-age quality and image semantic information. We have studied the characteristics of human visual system. We find that when observing images people will first focus on the semantic areas, and the change in semantic areas will affect human perception of image quality. Our experiments on convolutional neural networks show that image dis-tortion will affect the semantic obviousness and semantic area of images.(2) In this paper we propose a new image quality assessment framework based on the combination of high-level semantic information and low-level characteristics. We extract all object proposal regions from images. The detection scores of these regions is our high-level feature which reflect the semantic obviousness. From these regions some raw patches are extracted and then encoded to a visual codebook. The encoded vectors of these regions are pooled to get the local feature. Low and high level features are fused and fed into a support vector regression model to get the final quality score.The proposed approach is demonstrated to be superior to the existing image quality algorithms. Cross-dataset experiments show the generalization ability of our method. The integration with our framework obtain obvious performance improvement of other algorithms.