Feature Aggregate And Data Driven Based Blind Image Quality Assessment

As the most straightforward information carrier,digital images contain massive valuable data.In the procedure of the whole lifetime of a ditial image:acquisition,transmission,compression,storage and all kinds of image processing,being affected by imaging system and imaging environment,image compression algorithms,image transmission channels,image storage medium and diverse image processing algorithms,images' perceptual quality may undergo distortions with different types and levels and then cause losses of information included in the digital image.Being able to recognize and quantize an image's quality level is vital to control,maintain and enhance its quality,hence,conducting researches on image quality assessment has profound and lasting practical significance.Furthermore,image quality assessment can also serve as objective performance criteria for other image processing domains such as image denosing,image deblurring,image restoration,image super resolution restortion and image enhancement.In realistic application,due to the absence of reference information in most scenarios,blind image quality assement has more research value than full reference and reduced reference image quality assessment.This thesis focuses on blind image quality assessment and contributes to it from four aspects.This thesis firstly proposes a multiple channels aggregated features quality evaluator(MCAF)by extracting diverse heterogenous and complementary quality-aware features group which similuate center-surround antagonism mechanism in HVS from multiple color spaces to acquire the description of image information along with a novel image representation,which addressed both the problem of insufficience in utilization of features and deficiency in image representation existed in traditonal blind image quality assessment models.Experimental results on three mainstreaming synthetic image quality assessment databases manifest that MCAF achieve higher consistence with subjective assessment than other classical models and good generalizability.The second conritbuion of this thesis is the propose of a novel model named SE-ML-MI,aiming at quality assessement of authentically distorted images.Represented by convolutional neural network,deep learning technology overwhelm many image processing and computer vision domains in recent years.This thesis proposes a novel model named SE-ML-MI,which employs a relative new network SE-ResNet-50 as the basic feature extractor.Extending the idea of combing multiple heterogenous and complementary features as used in MCAF,it jointly extracts image low-level features,texture features and semantic information from shallow,middle and deep layers from the convolutional neural network.To deliver a better image representation,this theis simultanesouly utilize three simple yet effective pooling methods to aggregate features in each layer and then uses a dimenionality selection technology based on mutual information to alleviate the problem of high dimensional image representation and high redundancy caused by concatenating features from multiple layers by multiple pooling methods.Experimental results on two authentic image quality assessment databases and one multiply synthetic image quality assessment database manifest that compared with other classical models,SE-ML-MI can handle image quality asseesment task on authentic scenarios along with good generalizability.Meanwhile,it achieves a good trade-off between predction presicion and computational complexity.The third contribution of this thesis is it proposes an end-to-end learnable deep convolutional neural network DB-CNN.By combing two network branches with different advantages,it can handle well on both synthetic and authentic image quality assessment tasks.The first branch of DB-CNN is for synthetic distortion,named S-CNN.It uses a large-scale synthetically distorted image database to obtain good intialization weights through a classification task.DB-CNN's second branch is for authentic distortions,which is the VGG-16.Because VGG-16 has been pre-trained in the ImageNet which contains many authentically distorted images,it is able to capture characteritics of authentically distorted images well.By tailoring S-CNN and VGG-16 and combine them into an end-to-end learnable organized whole using bilinear pooling layer,DB-CNN achieves good experimental results on three synthetic image quality assessment databases and one authentic image quality assessment database.Futhermore,DB-CNN is shown to has good scalability,generalizability and robustness through experiments on the large-scale database Waterloo Exploration and the group MAximum Differentiation game(gMAD).Finally,this thesis introduces blind image quality assessment into the new-old sorting application of paper currencies.First of all,a large number of real paper currency samples were collected,which is composed of tow categories:the large-scale pre-training set and the subjective testing set.Then a light convolutional neural network is designed to hande new-old sorting application,which comprehensively adopts multiple philosophies for the designing the structure of the network.Furthermore,a two-stage training strategy,i.e.,pre-training and fine-tuning is adopted to take advantage of large-scale samples and avoid the problem of subjective labels at the same time.Experimental results manifest that this model has good discriminability and prediction stability.Ablation experiments also validate the rationality of the model designing.