Image Quality Evaluation Deep Learning Model Based On Multi-feature Fusion

Under the background of rapid development of modern science and technology,people's requirements for images are getting higher and higher.High definition and high quality are the basic goals pursued by digitalization of modern visual information.However,in the process of image acquisition,processing,transmission,and storage,due to some insufficient factors,such as collection methods,processing methods,transmission media,and storage devices,coupled with a series of object movement and noise pollution during transmission,etc.It can cause a certain degree of distortion to the image.Therefore,the correct evaluation of image quality is very important in practical application,and the application of image quality assessment(IQA)is extensive,which has a profound impact on the development of related industries,such as personal services,scientific research,and engineering applications.In traditional IQA models,various visual features are artificially extracted to predict visual quality of images.However,in deep learning manners,deep features are auto-learned via Convolutional Neural Network(CNN)models.So,it is a meaningful study to investigate how to fuse these two kinds of features for a better IQA model.In this paper,we propose a novel multi-feature fusion CNN model(MFNet)for IQA,which takes both artificial features and deep features into account.Some basic visual features are artificially extracted and then fixed in the CNN model,such that the CNN-learned features can further extract more focused information of Human Visual System(HVS).In the proposed model,excepting the extraction of visual features in the learning procedure,we proposed an algorithm that uses visual saliency to redistribute the subjective scores of image patches in the training set,and gradient features as the pooling strategy to fuse scores of all patches together.Hence,the proposed model is named as SG-MFNet.Finally,we trained the model on the TID2013 dataset and conducted experiments on the other three datasets to verify the accuracy of the model.The innovation of the model in this paper is:(1)The model can learn information of distorted images from both hand-crafted features and CNN-generated features.The roles of these two kinds of features in representing the visual quality are auto-determined via the proposed MFNet.(2)This paper redistributes the score of patches in the training set based on the saliency features of the image.The proposed three-region division of saliency map prevents large score variation of patches in the same image,while assigning more reasonable scores to each patch.(3)When getting the local score map after the learning,different weighting coefficients are assigned to different regions in the map according to gradient features of the test image.The edge regions have higher weights which correlates well with characteristics of the HVS.