Multi-level Feature Fusion And Multi-task Learning Based Image Quality Assessment Research

In the course of image transmission and storage,the problem of image quality degradation is unavoidable,which is challenging for subsequent image processing.In order to avoid inputting low-quality images into the image processing system,it is necessary to develop an automatic evaluation mechanism for image quality,which is significantly valuable for the research and application in image engineering and related fields.As a key branch of the image quality assessment research,fullreference image quality assessment(FR-IQA)can obtain a relatively higher accuracy rate.In this paper,we address the FR-IQA problem in the framework of deep learning,which is capable to represent the image feature in various semantic levels.However,the most of available methods have two limitations: 1)Do not make full use of image features extracted from the middle or lower level of the network.2)Ignore the effect of image distortion type on human eye visual perception.Based on these,we propose two research emphases: the multi-level feature fusion and multitask learning for full-reference image quality assessment.The main contributions are as follows:(1)We propose an end-to-end multi-level feature fusion based fullreference image quality assessment method.Previously,the researchers only used the high-level semantic feature,extracted at the last layer of the network,as the input for the score of regression calculation.However,those methods ignore the effect of middle-level and low-level features on image quality assessment.The proposed multi-level feature fusion method in this paper,with the high-level and middle-level features simultaneously input to score regression calculation part,takes the whole useful information of multiple levels into account.The Pearson Linear Correlation Coefficient(PLCC)of the proposed algorithm on the LIVE dataset achieves the prediction accuracy up to 98.3%.(2)Considering the effect of the distortion type of natural image on image quality assessment,we develop a multi-task learning based fullreference image quality assessment method,which use the previous framework we proposed as a sharing network to estimate the quality score and distortion type.Meanwhile,the two tasks are weighted by a Bayesian uncertainty method.The prediction of image distortion type and quality score provides a more comprehensive description for the distortion properties of an image,so as to improve the accuracy of image quality score prediction significantly.The experimental results demonstrate that the algorithm achieves a PLCC value up to 98.5% on the LIVE dataset,which has an improvement of 0.2% comparing with our first work.The proposed methods in the paper: take the middle-level features into the calculation of the assessment score for first time,and provide quality evaluation by making use of distortion type in multi-task learning.The methods proposed in this paper could be able to achieve high prediction accuracy,which provides novel and constructive ideas for the network structure of FR-IQA,as well as for the relative researches of image processing.