No-Reference Image Quality Assessment Based On Visual Cognition

As a richer information carrier than speech and text,visual data is the main way of humans to obtain information and understand the world.Taking and sharing high-quality images has become an indispensable part of our daily life.High-quality images are the basis to bring comfortable visual experience and transmit clear and complete information.However,due to the limitation of lighting conditions,the defects of imaging system,the loss of transmission compression,the mismatch of display device,coupled with the lack of photographic skill and the movement of object,etc.,distortions are inevitably introduced to images during the actual acquisition,processing,transmission and display processes.How to establish automatic and accurate no-reference image quality assessment methods is of great significance to guide image processing,optimize image algorithms and improve visual experience.The thesis aims to explore no-reference image quality assessment methods that conform to the process of human visual cognition,by referring to the relevant mechanisms of human visual system employed to perceive and understand visual signals.Specifically,the proposed methods and the main contributions proposed in the thesis are as follows:A traditional no-reference image quality assessment method based on spatial correlation of pixels is proposed in the first part.In the method,the neighborhood co-occurrence matrix is introduced as a novel structural descriptor.The neighborhood co-occurrence matrixes constructed from multi-channel and multi-scale of distorted image is used to highlight the quality perception information hidden in the relative position of pixels.Moreover,a series of statistic features including entropy,contrast and homogeneity are designed to measure the unnaturalness of the neighborhood co-occurrence matrix distribution for quality prediction.Finally,the statistic features of neighborhood co-occurrence matrix is combined with others natural scene statistics,and then a regression mapping from joint features to image quality scores is established.Extensive experiments demonstrate the proposed method has achieved significant improvements on distortions associated with color and locality.A deep no-reference image quality assessment method based on automatic data augmentation is proposed in the second part.To ease the limitation of insufficient images labeled with subjective quality scores,an auto-augment strategy based on aesthetic composition and visual attention mechanism is designed in the method.The strategy is able to determine the cropping size and cropping position adaptively to obtain several image patches with the similar quality from a distorted image as training data.Because of image quality mainly depends on saliency areas,the data augmentation process is guided through two steps in the strategy: 1)the cropping size is guided by the principles of aesthetic composition,while cropping most visual subject while leaving enough room for augmentation;2)adopt visual saliency detection as secondary screening to ensure that all image patches from the same distorted image have taken most visual saliency areas of the image and share similar image qualities.Finally,quality scores are regressed from multilayer convolution features extracted from auto-augment image patches by convolutional neural network.Experiments show that the algorithm has achieved outstanding performance on images with both synthetic and authentic distortions.A deep no-reference image quality assessment method based on internal generative mechanism is proposed in the third part.Human visual system not only process incoming visual signals locally,but also exists the internal generative mechanism.The method attempts to mimic internal generative mechanism of human visual system through a recovery and reconstruction network.Specifically,image is super-resolution reconstructed after down-sampling to mimic that human visual system derives and recovery the visual scene to be recognized based on the memory in brain.The difference between the restored image and the distorted image reflects the integrity of the details and structure information in the image to a certain extent,which can be used for image quality prediction.Then,a Siamese network is used as a quality assessment network to extract quality-aware features both from the distorted image and the restored image,the dual-channel features from where are employed to regression predict the image quality score.Experiments show that the method has achieved outstanding performance on most benchmarks,further improving the subjective consistency of no-reference image quality assessment.