Research On Multimodal Fine-grained Sentiment Analysis Based On Deep Learning

As a classical direction in the field of natural language processing,fine-grained sentiment analysis has been extensively and deeply explored in recent years.But in real life,the content people released on social media has been transformed from a single plain text modal information into multimodal information.Previous fine-grained sentiment analysis models consider only the text content,unable to model the multimodal information,and thus cannot accurately predict the sentiment polarities of opinion targets.Multimodal fine-grained sentiment analysis is a new research direction proposed recently.This direction aims at using multimodal information such as text and image to predict the sentiment polarities of opinion targets,so as to solve the problem that the previous fine-grained sentiment analysis models cannot model multimodal information.In this paper,multimodal fine-grained sentiment analysis is taken as the main research content.From the two perspectives of image information enhancement and multimodal interaction,the following three aspects are carried out:(1)A multimodal fine-grained sentiment classification algorithm based on graph convolutional neural network and multimodal LSTM is designed.In order to construct the expression of the overall information of the image,this algorithm uses the graph convolutional neural network(GCN)to model the interaction between objects in the image.In addition,a multimodal LSTM based on opinion targets guidance is designed to filter the image information,which can reduce the image noise and enhance the connection between opinion targets and image.Experimental results show that the algorithm can effectively model the relation between objects in the image,filter the irrelevant noise in the image,and obtain better performance than other models.(2)Multimodal fine-grained sentiment classification algorithm based on multimodal interactive Transformer is proposed.This algorithm enhances the semantic representation of images by extracting semantic information of objects in images and designs auxiliary tasks to reduce the semantic gap between text and images.In addition,the algorithm also proposes a neural network model based on hierarchical multimodal interactive Transformer structure to model the interaction among text,image and opinion targets.Experimental results show that the algorithm achieves the best results in Accuracy and Macro-F1.(3)Based on the existing problems in practical application,an end-to-end multimodal finegrained sentiment analysis algorithm based on joint training is proposed.The algorithm is designed from the perspective of joint training,and the image information is used to realize the two tasks of opinion targets extraction and sentiment classification simultaneously.In addition,adjective-noun pairs are used to construct a multi-perspective representation of the image to enhance the sentiment semantic information in image.Experiments show that this algorithm can effectively extract opinion targets and predict sentiment at the same time,and it is superior to other benchmark models on the index of Micro-F1.