Deep Representation Integration For Image Aesthetic Assessment

With the rapid popularization of cameras and smart phones,the amount of images and videos is dramatically increasing.The demand for high-quality aesthetic images is also growing.Image aesthetic assessment aims at computationally evaluating the quality of images based on artistic perceptions.Image aesthetic quality evaluation has wide applications in photography,face beautification and makeup,e-commerce,graphic design.etc.and has become one of the research focuses.Existing image aesthetic evaluation methods are designed for general images and face images,respectively,because there are specific laws for facial beauty.In this paper,we conduct research on these two topics distinctly,and propose one deep representation integration model for each topic,respectively.First,we propose one multi-level feature integration framework for universal image aesthetic quality evaluation.Specially,we propose to extract low-,middle-,and high-level features from a Convolutional neural network(CNN),and integrate them together for aesthetic prediction.To evaluate its effectiveness,we build three multi-layer aggregation networks(MLAN)based on different baseline networks,including MobileNet,VGG16.and Inception-V3,respectively.Experiments are conducted on the AVA database.The experimental results show that aggregating multi-layer features consistently and considerably achieved improved performance.Besides,MLAN outperforms previous state-of-the-art by 0.05 in accuracy.Second,we propose a co-attention learning mechanism to integrate the pictorial and composition information of a face for facial attractiveness prediction.Specially,the composition of a face is represented by parsing masks,i.e.pixel-wise labelling masks.We use a branched CNNs with the parsing masks and face image as the input,respectively.Besides,we employ a spatial attention mechanism to extract the significant locations in image,and a channel attention mechanism to select critical components among the masks.Finally,the features learned in both branches are integrated for facial attractiveness prediction.Experimental results show that the proposed method outperform previous state-of-the-art methods by 2.5 percents and 2.6 percents in terms of prediction accuracy,on the SCUT-FBP5000 and CelebA databases,respectively.Such superiority demonstrates the effectiveness of the proposed framework.In summary,the proposed representation integration frameworks are novel and have proven effective on standard databases.This work is therefore valuable for the development of image aesthetic quality assessment.