Blind Image Quality Assessment Based On Semantic And Hierarchical Feature Fusion

No Reference Image Quality Assessment,also known as Blind Image Quality Assessment(BIQA),refers to the Quality Assessment of distorted images only according to their own characteristics in the absence of original Image reference and Image feature reference.The aim is to enable computers to automatically and accurately perceive image quality as humans do.As a kind of IQA method with the highest practicability,BIQA is the key technology and important research content of automatic judgment and capture of high-quality images in computer vision and image processing.Blind image quality assessment for authentically distorted images has always been a challenging problem,since images captured in the wild include varies contents and diverse types of distortions.The vast majority of prior BIQA methods focus on how to predict synthetic image quality,but performance degrades when applied to real-world distorted images.In order to cope with this challenge,a blind image quality evaluation model based on semantic and hierarchical feature fusion is proposed to predict image quality.The IQA model is divided into three stages:feature extraction,perception rule establishment and quality prediction.The process of human perception of image quality is simulated:(1)In view of the different feature representation of different distortion in different color space,the perceptual distortion feature of multi-color space is introduced.The complexity of convolutional neural network structure and the nonlinear characteristics of network mapping ability are used to learn distortion features to ensure the accuracy and diversity of feature extraction from distorted images.(2)Judging whether the image is distorted or not is based on the real expression of the image,so understanding the image content helps to perceive the quality of the image.Therefore,the semantic information of the image is introduced to learn the image content,and the quality perception rules are formulated by the parametric network according to the semantic features of the image.(3)Considering the correlation between the features extracted from each color space and the distortion characteristics of the image itself,the weight of feature vectors extracted from different color space is allocated adaptively using the semantic features of the image.The fused image features are used as the input of the image quality prediction network,and the quality prediction is carried out according to the parameters provided by the parameter network,so that the image quality can be estimated in an adaptive way,which can be well extended to different distorted images captured in the real world.(4)Since the deep neural network is very easy to overfit,in order to improve the generalization ability of BIQA model,R-Drop regularization is added into the quality assessment network for the first time in this paper.The constraint terms are used to adjust the degree of freedom of the model to ensure the accuracy of prediction and improve the generalization performance.Experimental results show that the proposed method achieves superior performance on three synthetic image databases and two challenging authentic image databases,and achieves the best performance in the comprehensive ranking of IQA model performance indicators.The effectiveness and generalization of the proposed algorithm are further verified by experimental analysis,which provides a broader application prospect for IQA tasks.