Research On Visual Sentiment Analysis Based On Deep Learning

As a platform for users to create and share information,social media has become an important part of people's lives,then more and more people release massive multimedia content through social media every day to express their views and sentiment.The sentiment analysis of these user generated data can effectively analyze user behavior and psychology and explore user needs,which has important application value.With the increasing number of visual content released by users in social media,visual sentiment analysis has attracted wide attention.Different from the task of object recognition,image sentiment recognition is a more abstract task,the key of which is to extract discriminative visual features.Deep learning method can automatically learn features from large-scale data and is widely used in visual tasks.At present,many achievements have been achieved by combining deep learning with visual sentiment analysis,but there are still some limitations.For example,the existing methods usually extract features from the whole image but fail to fully consider the important local areas of the image;the existing methods fail to fully utilize the multi-level features of the image and fail to fully mine the spatial and semantic information of the features.In order to obtain more discriminative feature representation,this paper mainly uses the important local area information to improve the image representation and make full use of the complementarity of multi-level features to strengthen the features.Finally,the extracted discriminative features are used in visual sentiment analysis.To deal with the above problems in visual sentiment analysis,main research contents of this paper are as follows:(1)Aiming at the problem that the existing visual sentiment analysis methods mainly focus on the whole image to construct the sentiment representation,but do not treat the important local areas differently in the image,a visual sentiment analysis method of the integration of the global and the local features of image is proposed.Specifically,this method integrates region detection and sentiment classification into a unified framework,extracts the salient region features of images through the saliency detection network,and extracts the affective region features of images through weak supervised learning by using image level sentiment labels.Finally,the features of salient region and affective region are used as the local representation of image sentiment,and the local and global features of the image are fused to generate the more discriminative feature representation of the image.(2)Aiming at the problem that the existing image sentiment classification method based on deep learning fails to make full use of the multi-level features of the image and the expression ability of the features is insufficient,a visual sentiment analysis method based on multi-level feature fusion of dual attention is proposed to obtain more discriminative visual features,so as to improve the effect of sentiment classification.Specifically,this method first extracts multi-level features of image multi-channel by convolution neural network,then gives spatial attention weight to low-level features of multi-channel by spatial attention mechanism,gives channel attention weight to high-level features of multi-channel by channel attention mechanism,and enhances feature representation of different levels respectively.Finally,the enhanced high-level features are fused And low-level features to form discriminative features for training sentiment classifiers.