Emotion And Aesthetic Assessment On Social Images

With the development of social network and mobile device,it's convenient for us to share and acquiring social images.The incensement of the number of social images brings the higher level of requirements for automatic image processing,especially in the attribute of emotion and aesthetic of images.Nowadays,people tend to convey their emotional information to others through images.At the same time,they also want to enjoy or store the images with higher aesthetic quality in massive social images.Therefore,it is very important to study the emotion and aesthetic assessment of social images.The emotion and aesthetic are the high-level and abstract information of images,which contains lots of subjectivity.The research of emotion and aesthetic assessment of social images simulates human's high-level visual perception and psychological reaction by computer.It involves many disciplines such as psychology,art and computer,which are very challenging.The emotion assessment of social images is to assessment the human emotion attributes aroused by the images through the help of computers.The aesthetic study of social images is to assessment the aesthetic quality of images automatically by computer simulation.To solve such kind of research problems,a series of emotional and aesthetic features are designed for images.Then,these features are applied for emotion and aesthetic assessment by using machine learning algorithm or deep learning methods.Previous researches surveyed and analyzed a lot in image emotion and aesthetic assessment.They reduce the subjective influence of the emotions and aesthetics through the artist rules of images and proposed some effective methods,including traditional hand-crafted features methods and deep learning method.However,there are some problems in the existing methods of the emotion and aesthetic assessment in social image:1.The deep learning methods applied in emotion assessment of images less consider the prior knowledge of images,which influence the expression of image emotion a lot.2.The existing network structures used in deep learning methods are not fully applicable to the emotional assessment of images.The network structures do not sufficiently study the semantic features of images.3.It is not enough for the research of image aesthetic assessment.The traditional algorithms in this study cannot be effectively combined with the deep learning method.In this paper,we focus on the emotion and aesthetic assessment of social images.The research content of this paper includes: The multi-level deep convolution neural network for image emotion classification,The ResNet-GCN based image emotion classification framework,Aesthetic quality assessment of photos with faces.The details are as follows:1.The existing deep learning methods applied for image emotion classification do not fully consider the prior knowledge of images.In this paper,we proposed a multi-level deep convolution neural network for image emotion classification,which learn images from objects,color,local information levels for emotional characteristics to enhance the image emotion assessment performance.2.In order to learn emotion semantic information better for networks in deep learning methods,to make the network structure more suitable for the emotion assessment of images,we proposed the ResNet-GCN based image emotion classification framework.The use of GCN structure with large-size convolution kernels improve the network's learning horizons,so that the network are more effective to learn the emotional characteristics of the image to enhance the image emotion classification performance.3.In order to solve the problem of aesthetical assessment of social images,we proposes the new 78 dimension facial aesthetic features.Then,we combined them with the features learned by the deep learning method and the comprehensive aesthetics features of the existing research to enhance the aesthetic assessment performance.In the meantime,in order to apply the deep learning method in this research better,we construct a new aesthetic dataset with photos with faces.