Research On Early Rumor Detection Model With Multimodal Social Media Based On Hyper-Graph Theory

The popularity of social media in the mobile internet era has lowered the threshold for people to access information and understand social developments.However,in an increasingly convenient Internet environment,the cost of spreading false information has also been reduced,and the time from rumor generation to fermentation has been shortened,which can easily cause huge social harm.But if we rely solely on manual review and refutation,we can not cope with the suddenness and dispersion of rumors.Therefore,studying automatic detection schemes for rumors has practical significance.The challenges of early multimodal rumor detection are as follows:social media content is complex and noisy,different posts contain different information richness,analyzing each post independently may not yield valuable features.Moreover,the text and image features of posts need to be associated with finer-grained and higher-order semantics,rather than simply stitching together.Based on the above problems,this paper proposes a multimode rumor detector model based on hypergraph theory(Multimodal Hypergraph Fusion Rumor Detector,MHFRD).This paper first analyzes the characteristics of rumor datasets in social media,and on this basis uses BERT and Vision Transformer to extract text and image features of posts.In view of the fact that there is little information available for early rumor detection,this paper uses hypergraph to model posts and the relationship between posts.In the features fusion stage,hypergraph based on text modality and image modality are constructed respectively,and the hypergraphs of the two modalities can be merged into a more complex hypergraph by using the incidence matrix of the hypergraph,so as to complete the mapping of heterogeneous features to the same feature space and modals fusion.The experimental results show that the model proposed in this paper is better than att-RNN,EANN and other multi-modal rumor detection methods in terms of accuracy,precision and F1-score have improved.The innovations of this paper:First,this study not only considers the features of the post itself,but also considers the relationship between posts.The hyperedges of hypergraph have the ability to describe high-order relationships,so this study uses hypergraph to model the relationship between posts and posts.Second,this study uses the scalability of hypergraph incidence matrix and the ability of hypergraph convolution to capture higherorder relationships between samples to design a multimodal fusion mechanism based on hypergraph theory to enhance the modeling and prediction ability of the model.Third,image text inconsistency is one of the characteristics of rumors.Therefore,in addition to classification loss,this paper also uses the similarity loss function based on image text consistency.