Research On Image Aestetic Quality Assessment Methods

Image aesthetic quality assessment aims to automatically measure whether an image looks beautiful in human's perception,and has recently received increasing attention and been a hot research direction in the field of computer vision and multimedia due to its wide range of applications.Image aesthetic quality assessment has many applications,for example,it is expected that modern image search engines would rank results not only by topical relevance but also by aesthetic quality.The returned images are not only relevant to query,but also pleasing.Designing an automatic image aesthetic quality assessment system poses many challenges.On the one hand,the factors that determine the aesthetic quality of an image are complex.It is hard to design one or more features that clearly represent these factors,such asimage composition,lighting condition,usage of color.On the other hand,aesthetics is essentially a subjective human perception.As the saying goes,"beauty is in the eye of the beholder",and different people may have different ideas about the beauty of the same image.In most existing studies,image aesthetic quality assessment is transformed to a classification or regression problem,where each image is assigned a single label(i.e.,category or score)indicating its aesthetic quality level.In light of this,a single label is insufficient to characterize the disagreement among users'aesthetic perceptions effectively.In this paper,we propose to use a distribution to depict the aesthetic quality of an image.Each component of the distribution indicates the probability of users assigning a specific quality level to that image.This form of distribution is able to characterize the disagreement among users' aesthetic perceptions.In addition,depending on practical needs,a distribution can be easily converted to a category or score with its numerical characteristic,such as the expectation and variance.This ensures that the traditional tasks of aesthetic classification and ranking can be smoothly performed with the distribution representation as well.Both of the works described in this paper apply a distribution to depict the aesthetic quality of an image.The second chapter introduced a work based on label distribution learning.This work seeks to learn the mapping from an instance to its distribution over multiple labels in a supervised manner.Specifically,we adopt the Multivariate Support Vector Regression(M-SVR)as the backbone of our learning algorithm.Besides,we consider the number of rating users for each image are different.In fact,it is clear that the more users have rated an image,the more reliable is the aesthetic distribution.In view of this,we assign each training example a weight to reflect its reliability.Besides,we take account of the correlations between quality levels to further enhance the robustness of our approach.Recently,convolutional neural network(CNN)has brought in revolutions to a variety of applications in the field of computer vision.Convolutional neural network is an end-to-end architecture,and has strong ability at learning feature automatically which avoids the difficulty of designing hand-craft features.There are a lot of studies that are based on convolutional neural networks.However,a critical issue of using CNN for image aesthetics assessment lies in its requirement of a fixed input size(e.g.,224 x 224).Generally,the input images have to be fitted to the fixed size via warping or cropping,which may bring about the geometric distortion or the loss of the entire content.More importantly,fitting an image to the fixed size could cause severe damage to its intrinsic aesthetic appeal,and thereby affect the efficacy of the subsequent process of aesthetics assessment.To address the above challenge,we propose to realize image aesthetic distribution prediction with fully convolutional network(FCN),which is a variant of CNN adapting the fully connected layers into the convolutional layers.Because the fixed-size constraint of CNN comes only from the fully connected layers;the convolutional layers operate convolution with a filter in a sliding-window manner and do not require a fixed-sized input.Therefore,FCN allows for arbitrary-sized input images by using the fully convolutional architecture.Experiments on two benchmark datasets well verified the effectiveness of our approach in scenario of aesthetic distribution prediction.Besides,our approach also obtain better performance than traditional image aesthetic classification methods in scenario of aesthetic label prediction,which confirms the good generalization capability of our approach.