Ranking-based Image Aesthetic Quality Assessment

Image aesthetic quality assessment is a hot research topic in computer vision field,which can be combined with many practical applications and has great application prospect.Most existing methods define it as a classification task,they describe image aesthetic quality using binary aesthetic labels,namely "high quality" and "low quali-ty",and train classification models with aesthetic features and corresponding aesthetic labels.Due to that binary label is a coarse-grained expression form,this paper propose to focus on fine-grained relative aesthetic quality ranking.Despite that the proposed classification and regression models are able to handle aesthetic quality ranking task,they only utilize the absolute quality rather than relative ranking relationship,which leads to the poor performance.Some researchers propose listwise approach for relative aesthetic quality ranking.However,only limited suc-cess was achieved due that,1)the ranking order of all images is employed in training while images with different visual content are yet not reasonable to be compared,2)pre-defined features can't describe the aesthetic attributes well.To address these challenges,this paper first propose an image pair-based ranking framework.Image pairs are formed to capture the relative ranking relationship within training images,and ranking function is trained with these pairs.In addition,informative pairs generation strategy is applied to filter out noisy pairs.To further improve the quality of training samples,this paper propose a visual similarity-based approach to generate image pairs.A content based image retrieval system is first built to retrieve visual similar images,then query image and retrieved images are formed as image pairs under certain criterion.To get rid of the limitations of pre-defined aesthetic features,a dual-channel deep neural network is constructed as the ranking model,which takes training pairs as input and ranking re-lationship as optimization objective.A corresponding pairwise ranking loss layers is designed to calculate the ranking loss and the network gradient.During testing,the ranking scores can be predicted and are only used for ordering images,the absolute values of which are meaningless.The proposed strategies are evaluated on two large-scale public datasets,AVA and CUHKPQ,and are compared with several other methods as well,the experimental results demonstrate the superiority of proposed strategies on aesthetic quality ranking task.