Research On Image Quality Assessment Method Based On Multi-Task Learning

With the rapid development of Internet technology,people rely more and more on the Internet to get the information they need.According to incomplete statistics,video content accounts for about 90% of total Internet traffic,and in the rapidly growing mobile Internet,the proportion of video traffic is also as high as 64%.Images and videos are widely used in many application scenarios for their intuitiveness,convenience,and rich information content.Various distortions occur during the generation and distribution of images and videos,which can degrade the quality of the image and thus affect the user experience.Therefore,how to evaluate the quality of images has become an important task in the field of computer vision,which has very broad application prospects.Currently,deep learning-based image quality assessment methods are the mainstream in the field of image quality assessment,and such methods often require a large amount of annotated data for training to achieve the desired results.The existing image quality evaluation datasets are small in size,and it is costly to produce large-scale image quality assessment datasets,so the models always suffer from overfitting and insufficient generalization performance.Multi-task learning can improve the generalization ability of the model by using the relevant information implied in multiple related tasks,which can alleviate the problem of insufficient labeled data.Based on this,this paper proposes three multi-task learning-based methods for no-reference image quality assessment,and the main work and innovations are as follows:A multi-task self-supervised no-reference image quality assessment method for predicting distortion types and quality scores is proposed.First,a large number of distorted images were synthesized,and the synthesized distorted images were scored by the full reference method MDSI,so that the synthesized distorted images obtained the corresponding distortion type labels as well and distortion score labels;Subsequent multi-task pre-training with prediction of full-reference MDSI scores as well as distortion types as two subtasks enables the model to learn semantic features useful for downstream tasks from a large number of distorted images;Finally,fine-tuning is performed on different no-reference tasks so that the semantic features learned from multi-task pre-training are migrated to different no-reference tasks.The test results on several datasets show that the method can achieve good performance on the no-reference task and alleviate the problem of insufficient model generalization performance due to insufficient labeled data.(2)An improved MMo E multi-task self-supervised no-reference image quality assessment method is proposed.The method builds on the work in the previous chapter by using MMo E as a feature extraction network and improving it to be more adapted to image quality evaluation.In addition,a multi-viewpoint feature fusion method is also proposed in conjunction with the proposed network structure to further enhance the global sensing capability of the model.Experimental results on several datasets demonstrate that the method in this chapter achieves better image quality assessment performance compared to the method in the previous chapter.(3)A multi-task no-reference image quality assessment method based on visual perceptual characteristics is proposed.First,a subtask for predicting quality levels is designed to be applicable to both synthetic and real distortion datasets,taking into account the characteristics of human visual perception;Subsequently,a multi-task learning network applicable to image quality assessment is proposed based on the use of Vi T as a feature extraction network;Finally,multi-task learning is performed using the proposed network structure with predicted quality levels and predicted quality scores as subtasks.The experimental results show that the method in this chapter can achieve good performance on both synthetic distortion datasets and real distortion datasets without pre-training.