Image Quality Assessment Based On Visual Computing And Human Perception

Vision is one of the most vital senses for most living creatures on this planet. To human, it accounts for over 80 percent of information we acquire every day. It helps us survive from the environment and understand the world we live. Since the camera was invented, human society has stepped into a visual information era. Images have become one of the most widely used yet efficient media in our daily life. Image quality consequently concerns both the academic and the public. However, the image is easily and inevitably noised within the procedures of acquisition, transmission, processing, storage, etc. The distorted image not only cripples the image processing system, but also hinders human observers from perceiving. As a result, the need of accessing image quality, subsequently optimizing relevant systems, has been raised.Based on human perception, this thesis aims to: 1) provide a profound insight into the research of image quality assessment(IQA); 2) offer computational models that correlate well with human observers. Specially, in the thesis, the human visual system(HVS) is systematically investigated to look for the contributing factors of quality perception. Many HVS characteristics are analyzed to determine the underlying perception mechanism. As a result, several image quality metrics are developed by incorporating the qualitative evaluation, visual attention mechanism, unsupervised feature learning, etc. Majority of them are first time presented in this research area, which is hoped that those findings could shed a light on the future works of this field.The models and their major contributions are outlined as follows:1) A fuzzy classification based reduced-reference IQA approach is proposed. Inspired by the human’s qualitative perception of image quality, as well as the fuzzy theory, we develop a reduced-reference image quality metric especially for communication scenario. In this work, five fuzzy sets are designed to partition the image quality space. Each image belongs to all of them with corresponding degrees of membership. A wavelet domain-based natural scene statistics feature is extracted to facilitate the training of fuzzy classifier. The experiment shows that the proposed work correlates well with the human evaluations and outperforms the state-of-the-art reduced-reference metrics.2) A deep learning framework for blind IQA is presented. Human visual system bears complicated structures and thus leads to extremely nonlinearity. The conventional machine learning fails to ascertain such nonlinearity due to its insufficient depth. This work adopts deep learning network, and seeks to investigate how to blindly assess image quality by learning rules from linguistic descriptions. Specially, the discriminative deep model is trained to assign images to five explicit mental concepts, i.e., “excellent”, “good”, “fair”, “poor” and “bad”. Natural scene statistics features are fed into the deep network. An innovative quality pooling is designed based on Bayesian’s rule to convert the qualitative labels into numerical scores for general utilization and faire comparison. The classification framework is not only more natural than the regression based models, but also robust to the small sample size problem. Thorough experiments are conducted on popular databases to verify the model’s effectiveness, efficiency and robustness.3) Information divergence is introduced for visual saliency detection. Visual saliency detection allows human visual system to reduce redundant information and emphasize the visually important regions. It is involved in quality perception, and thus regulates image quality assessment. This proposed work is based on a hypothesis that information divergence, i.e. the non-uniform distribution of information, gives rise to visual saliency. Based on this assumption, a two-stage framework for saliency detection is introduced. Sparse feature is learnt with the help of independent component analysis(ICA) and difference of Gaussians filter in the first stage. Bayesian surprise model is adopted and improved in the second stage to produce the information divergence across an image. Experiments are conducted on several publicly available databases. The results verify the effectiveness of the proposed method compared to the state-of-the-arts.4) A saliency-guided deep framework for blind IQA is proposed. Inspired by a fact that visual attention and language are natural ways for human to observe and describe the environment, respectively, we incorporate visual attention to better the no-reference image quality metric. A united saliency-guided deep framework is proposed in this work. Specially, with the help of information divergence model, the relation between images and linguistic labels is learnt upon the training data set. The Bayesian quality pooling is applied to convert the qualitative labels to numerical scores. Experiments are conducted on popular databases to verify the effectiveness and robustness of the proposed method.5) A saliency-guided feature learning scheme is devised for blind IQA. It is seen that almost all existing image quality assessment methods employ hand-crafting features. Such features need to be carefully designed and tuned in order to work properly, which is believed to be a major issue limiting the flexibility and versatility of computational models. To this end, a saliency-guided feature learning is presented in this work. The information divergence model is applied to extract salient patches from images, and then ICA generates filter banks upon the patches. The image-level feature is then generated by an encoding stage. The resulted feature is applied in the previous deep framework. The experiments are conducted on popular databases to validate the effectiveness of the proposed model.In the thesis, I seek to answer a very fundamental question: how we see the world. By incorporating the findings from many interdisciplinary subjects, such as neuropsychology, physiology, machine learning, etc., these proposed models attempt to tackle the question from different point of view. Starting with reduced-reference metric, marching to no-reference one, and going further to visual saliency-guided one, this thesis investigates the field thoroughly and systemically. It is hoped that the findings and ideas presented in this thesis could shed a light on the human perception based image quality assessment.