Dual Pattern Learning Image Emotion Recognition Based On Multi-features Representation

Image emotion recognition is a complex and challenge research topic,and it is usually used in human-machine interface and public opinion analysis.How to build the gap between features in images and emotion of human is the main problem in image emotion recognition task.In this paper,a dual pattern learning image emotion recognition based on multi-features representation algorithm is proposed.The multi-features representation algorithm is proposed in this paper to extract more abundant features to describe the image emotion.And the dual pattern learning network can learn the relationship between different features in images to bridge the gap between low level features and high level semantic information to achieve the goal that bridging the gap between images and human emotions.The main work is as follows:1.Build a large image emotion datasetLacking of cleaning labeled images data is a serious problem in image emotion recognition.In this paper,positive emotion images and negative emotion images are obtained from Flickr website using key words.And a semi-supervised method is used to obtain the accurate label.2.Clean the dataDue to'the relationship between images and corresponding labels is insensitive to the human when using our image emotion dataset or using common image emotion datasets,the problem of images which have weak relevance with corresponding labels exist in dataset.The Progressive CNN(PCNN)architecture is used to clean images data and selects the images which have strong relevance to their labels.3.Multi-features representationFeatures of images is an effectiveness method to describe the images.More abundant features of images can describe the emotions of images from various aspects.In the research we found that emotions of images relate a lot of factors.At first,the object in an image is included in foreground images which can provide high level semantic information.And the background images lack of object which can provide some basic features of images,such as color,texture features of images etc.The algorithm of multi-features representation is proposed in this paper using different CNN architectures to extract different level features to describe the image emotion.4.Bridge the gap between images and emotionFeatures of images is an useful method to describe the emotions of images,but features of images must be integrated and analyzed due to that the image emotion recognition is more subjective.So there is a huge gap between the images and emotions needing to be bridged.The dual pattern learning networks is to simulate the mechanism that people can process two images at the same time,and it is used in this paper which learns the similarity or difference in two images to build the gap between features of images and emotions of human.Through learning robust features between different level information can improve the accuracy of emotion recognition effectively,and bridge the gap between images and emotions of human beings.This paper based on the above four aspects to study.The experimental results based on Twitter2,ArtPhoto and Flickr LDL datasets show that the PCNN can select the data which have strong relevance to labels effectively.The multi-features representation algorithm provides more abundant features of image to improve the problem that using single features describes the emotion of images.At last,the dual pattern learning network is used in this paper can help to build the relationship between image features and emotion of human.Through the methods and algorithms are proposed above,the accuracy of image emotion recognition has improved.