Research On Image Emotion Classification Method Based On Emotion Source Self-attention Interaction Fusion

With the advent of 5G era,social media such as Sina and Douyin have also achieved unprecedented development.Internet users have become accustomed to using pictures and videos to express their emotions and attitudes.In this way,it has become a research hotspot in the field of computer vision to obtain the potential value in these multimedia data,understand the emotions hidden behind the images,and predict the user’s behavior through these data.Previous studies can be seen that the main contradiction image emotion classification task has two,one is the low level features and high level of emotional "emotion" gap exists between the two is most of the advanced features of image emotional classification methods only from the global perspective of the whole image feature extracting,ignoring the contribution of local area for mood for image.In order to solve the above two problems,based on cognitive psychology,the visual factors that induce human emotions in images are called emotional sources.According to the different objects that induce emotions,the emotional sources are divided into subjective emotional sources and detailed emotional sources,and the subjective emotional sources are further divided into global emotional sources and significant emotional sources.Then,Image emotion classification method based on emotion source self-attention interaction fusion is proposed,which comprehensively considers the influence of subject emotion source on image emotion and the influence of correlation between detail emotion source on emotion.In this model,the full convolutional neural network was used as the detection model of significant emotion sources,and the regions of significant emotion sources were extracted.The global emotion source region and significant emotion source region were fed into the visual representation extractor to obtain significant emotion source feature and detail emotion source feature.Compared with the emotion classification method based on single emotion source,this method utilizes the emotion information of subject emotion source and detail emotion source,and has better robustness to various images.Extensive experiments have been carried out on a number of large-scale image emotion classification data sets and the model shows excellent performance.