Research On Sentiment Analysis Of Image And Text Fusion Based On Deep Learning

Sentiment analysis initially originated from natural language processing,but with the vigorous development of social media tools such as Weibo and We Chat,there are not only texts on the Internet,but also multimedia information such as image,speech and video.Analysis of this information can effectively help public opinion monitoring,personalized recommendations,etc.This paper mainly studies the sentiment analysis of image and text fusion,the main work is as follows:(1)Research on the influence of salient and face object on image sentiment expression.This paper proposed an image sentiment analysis method based on multi-visual objects fusion.First,the salient object and face object area are detected in the whole image;Then,The feature pyramid is used to improve the CNN to recognize the emotion of the salient object,and the weighted loss CNN is constructed to identify the emotion of the face based on the multi-layered supervision module;Finally,the salient object sentiment,facial object sentiment and the sentiment directly recognized by the whole image are fused to get the final sentiment classification result.The experimental results show that the sentiment analysis based on multi-visual objects fusion is more accurate than the method of directly identifying the whole image.(2)Research on text sentiment analysis methods.Existing text sentiment analysis methods pay less attention to high-level semantic features and low-level word vector sentiment features,and interactive features between whole sentence and the local sentence,and proposes a multi-grain sentiment analysis method.Firstly,the BLSTM combined with the attention mechanism is used to obtain the high-level semantic features,and the low-level word vector emotional features are combined to obtain the sentence-level sentiment features.Then joint CNN and BLSTM,and extract the interactive sentiment features of the whole sentence and the local sentence by using the attention mechanism;Finally,sentiment recognition is carried out by combining the whole sentence feature,local sentence feature and interactive feature information.Experimental results show that sentiment analysis of multi-grain sentences achieves higher accuracy than sentiment analysis of whole sentences or local sentences alone.(3)Research on the image and text fusion method of sentiment analysis,and a image and text fusion sentiment analysis method of multi-visual objects and multi-grain sentences is proposed.Combining the proposed image sentiment analysis method based on multi-visual objects fusion and multi-grain sentences sentiment analysis method,a methodof sentiment analysis of image and text fusion is designed.Through the fusion of salient object sentiment features,face object sentiment features,whole image sentiment features,local sentence sentiment features,the whole sentence sentiment features,as well as the interactive sentiment features between whole sentence and the local sentence,the final image and text fusion sentiment polarity is obtained.The experimental results show that the method of sentiment analysis combining multi-visual objects and multi-grain sentences is effective and more accurate than the method of sentiment recognition based on image or text modality.