Research On No-Reference Image Quality Assessment Based On Dictionary Learning And Uncertainty Estimation

With the development of the Internet,the advancement of mobile communication technology and the popularity of smart devices represented by smart phones,digital iamges are playing an irreplaceable role as an important information carrier.How-ever,digital images inevitably introduce various distortions in the process of acquis-tion,compression,processing and transmission,and these distortions not only affect the human perception of image quality,but also influence the computer vision tasks.Therefore,it is important to automatically and accurately assess image quality.No-reference image quality assessment methods predict image quality without assessing the reference image.The high degree of proximity between no-reference assessment and subjective assessment make them have higher research value and wider application prospect.In recent years,the no-reference image quality assessment(NR-IQA)meth-ods based on convolutional neural network(CNN)have made great progress,but still face a critical problem:the lack of sufficient ground truth samples for training.Previ-ous CNN-based methods tackle this challenge in different ways.However,they all have own defects:(1)the image quality of any size cannot be quickly evaluated when inherit-ing the architectures and weights from pre-trained network follwed by fine-tuning with the original task input size(2)the image distortion type is not considered(3)the noise in image patches' labels by assigning the mean opinion score(MOS)of an image to all patches within it.To address the abovementioned problems,the thesis introduces dictionary learning and uncertainty estimation,and studies on deep convolutional neu-ral network based methods from the two aspects of model design and training methods.The content of this thesis are as follows:To address the problem that CNN based NR-IQA methods cannot predict arbitraty size image quality quickly,a NR-IQA method on dictionary learning and fully CNN is proposed.The algorithm uses a fully CNN to extract local features with more pow-erful representation ability.The fully CNN is a pre-trained network,thereby reducing the need of training samples with labels.Moreover,any number of local features can be encoded into a fixed length representation with the dictionary learning based local feature encoding module.Thus,the model can predict quality scores for input images of arbitrary sizes in a single forward step.Experimental results prove that the subjec-tive and objective consistency of the method surpasses other methods.In addition,the method is more than twice as efficient as the baseline.To address the problem that the model trained by the subjective quality score is not senstive to the image distortion type,a NR-IQA algorithm based on multi-task CNN is proposed.The multi-branch structure is used to decompose the task of rating image quality into two subtasks:namely distortion identification and distortion-level estima-tion,which reduces the difficulty of the task.A local feature encoing module based on dictionary learning is used to reduce the complexity of multi-task network design.Moreover,the label distribution learning method is adopt to unify the subtasks into classification problems,which reduces the difficulty of model training.The extensive experiments on benchmark datasets show that the proposed algorithm has better subjec-tive and objective consistency in image quality assessment,and the prediction accuracy of the distortion type is better than other methods.To overcome the noise in image patches' labels by assigning the mean opinion score(MOS)of an image to all patches within it,a NR-IQA based on uncertainty esti-mation and CNN is proposed.The algorithm uses the multi-branch structure to predict the image patch quality score and the uncertainty of its output.The uncertainty-based loss function is used to reduce the effect of noise on model training.In addition,the algorithm does not require label information of uncertainty and has a wider application prospect.Experiments show that the algorithm accelerates the convergence speed of the model while achieving better performance.