Research On Multimodal Sentiment Analysis Based On Attention Mechanism

Nowadays,with the rapid development of the Internet,people like to express their current emotions on various social media,when expressing emotions,the combination of text and pictures is more popular,accurately grasping the emotion of the target can improve the application value in the aspects of recommendation service,quality control,event analysis and so on.The task of multimodal sentiment classification is to identify the emotions expressed from people by extracting and integrating semantic information from these different modalities,each modality has its own emotional expression,and there are also correlations between modalities to provide complements for the final emotional expression.Therefore,how to make good use of the internal information of each modality and the interaction information between modalities is the key to multimodal sentiment classification task.This paper takes the multimodal data combined with words and pictures published by users on social media as the research object,and takes how to extract and integrate different modal information as the starting point to carry out the research of multimodal sentiment analysis.The main work of this paper is as follows:(1)Considering the incompleteness of single-modal sentiment analysis and the redundancy of fusion information in multi-modal sentiment analysis,a multi-modal sentiment analysis method based on attention mechanism and tensor fusion was proposed to improve the above problems.The core of this method is to introduce the attention mechanism to focus on the more noteworthy features in the text and image before tensor fusion of the extracted features of different modalities,firstly,the text features are extracted by the bidirectional gated recurrent neural network with the attention mechanism,and the image features are obtained by combining the residual structure and the CBAM attention mechanism,then,after the extracted text and image features are obtained,tensor fusion is used to capture the intersection between text and image features,finally,principal component analysis is used to further reduce the redundancy of multi-modal information and support vector machine is used to obtain sentiment classification results.Experimental results show that the proposed method can improve the performance of multimodal sentiment classification tasks.(2)In order to make the information between different modalities fully interact,and improve the connection between text and pictures,based on the previous multimodal sentiment analysis method,a multimodal sentiment analysis method based on cross-modal interactive attention was proposed.Firstly,the LDA is used to obtain the topic modal information in the text information,and the topic content is used as a new modal information to help deepen the relationship between the text and image features,secondly,BERT’s more excellent representation ability is used to extract text and topic features respectively,and the image features are extracted using the residual structure combined with CBAM attention mechanism,after extracting the three modal features of text,topic and image,the topic and image features were stitched,then,the text features and the image features combined with the topic interact with each other through the cross-modal interactive attention to obtain richer multimodal interaction information,finally,the multi-modal features were normalized by multi-layer perceptron and Soft Max to obtain emotional labels.Experimental results show that the proposed method can improve the accuracy of multimodal sentiment analysis more effectively.