Sarcasm Emotion Detection Based On Multi-Modal Features Fusion

With the rapid development of internet technology,social media and e-commerce have become an indispensable part of people’s daily lives.On these internet platforms,people use diverse ways to express themselves,among which sarcasm,as a rather special form of language expression,has gradually been widely used.Sarcasm refers to using artistic techniques such as metaphor and exaggeration to expose,criticize,and attack people or things,so that everyone can have a clearer and deeper understanding of these mistakes and weaknesses.However,due to the special nature of sarcasm,that is,the meaning and emotions expressed by the text are opposite to what they really want to express,sarcasm detection has become a challenging task in natural language processing.Traditional sarcasm detection methods mainly rely on analyzing text features.However,due to the limited information provided by text features,the accuracy of these methods is not high.In recent years,with the development of multi-modal information processing technology,sarcasm sentiment detection has started to focus on multi-modal information and has achieved some results.However,current research only focuses on incongruity information between text and image or between different text segments,and lacks detection methods that simultaneously consider both types of information.To address these problems,this thesis proposes a novel multi-modal sarcasm detection model MMCAN(Multi-Modal Co-Attention Network).The model consists of three main components: a multi-modal feature extractor,a co-attention transformer module,and a multi-modal feature fusion module.The multi-modal feature extractor is used to extract text features,image features,and text sentiment consistency features,while the co-attention transformer module captures relationships between modalities.Finally,the multi-modal feature fusion module combines the extracted features and modal relationships to achieve sarcasm detection.The model proposed in this thesis was experimented on the Twitter dataset,and the results show that the proposed model can effectively improve the accuracy of sarcasm sentiment detection.