Research On Social Media Fake News Detection Method Based On Fine-Grained Multimodal Feature Fusion

With the rapid and iterative development and upgrading of Internet technology,the emergence of self-media users has significantly impacted traditional media and gradually occupied major media platforms.As a result,the information acceptance habits of internet users are gradually being reshaped,and social media platforms,primarily microblogs and Twitter,have become the mainstream of public use.These social media platforms are inundated with all kinds of information,which contains substantial news value and caters to people’s demand for information.However,individual media organizations and self-media users deliberately fabricate news content,distort news facts,and tamper with original news pictures to garner public attention and generate revenue through fraudulent clicks.More seriously,fake news spreads unchecked on the Internet,causing significant damage to the order of cyberspace and creating a negative social impact.Therefore,it is crucial to explore an effective means of detecting fake news.With the development of diverse news forms,most of the existing news content includes multimodal information,such as text and images.As a result,researchers are increasingly seeking to detect fake news by fusing multimodal features.However,the existing work of detecting fake news based on multimodal information still faces several problems:(1)Most existing methods for detecting fake news based on multimodal information solely rely on deep semantic features,leading to a significant loss of effective information at the shallow level,such as image structure,color,and texture.This limitation negatively impacts the effectiveness of the model for fake news detection.Additionally,fusing features between different modalities using simple methods,such as direct summation or splicing,poses difficulties in handling the variability between cross-domain features.Furthermore,such methods fail to effectively fuse the essential information between modalities in a fine-grained manner.(2)Existing methods for detecting fake news based on multimodal information typically involve directly fusing semantic vectors of text and images.However,these methods lack correlation studies of fine-grained object relationships within and between modalities,leading to the model’s limited ability to perceive the authenticity of fine-grained objects.To address the above issues,a series of studies have been conducted in this thesis,which include the following two main aspects:(1)To address the issue of underutilization of shallow features in multimodal information,this thesis proposes a multimodal fake news detection model based on a progressive fusion network.The proposed model utilizes a Transformer structure as a visual feature extractor to capture the representational information of images at different levels.The model combines the features obtained by the text feature extractor and image frequency domain feature extractor at different levels and leverages a fusion module to achieve fine-grained fusion between different modal features in the same level.To better fuse cross-domain features,a hybrid fusion module based on a multilayer perceptron is proposed.This module maps different modal information into a unified domain through a fully connected layer and combines feature transposition operations to realize interaction between fine-grained information.Experimental results on real datasets show that the proposed model achieves 83.3% accuracy on the Twitter dataset,which is at least 4.3%better than other state-of-the-art methods.(2)To address the lack of exploration into the correlation between inter-and intra-modal features,this thesis proposes a multi-modal graph fusion-based method for fake news detection that can sense the correlation between fine-grained objects within and between modalities.The proposed method models the information of different modalities as a graph.For textual information,the semantics and single syntax of the overall text are embedded and represented as multiple nodes of the graph.The overall semantic features of the image and the syntactic features of individual objects within the image are also represented as multiple nodes of the graph.To efficiently learn information interactions within modalities in real-world data,the modal graphs are fused to extract more robust feature representations,which are then used to identify fake news.Experiments show that the proposed method achieves an accuracy of 87.2% on the microblogging dataset,which is better than the existing baseline method.In summary,the research of social media disinformation detection method based on finegrained multimodal feature fusion proposed in this study realizes the detection of true and fake news from two perspectives of progressive fusion network and graph fusion,and all experiments and data runs verify the relevant conclusions from real scenarios and realistic data,proving that using two social media modalities,text and image,is an effective verification path to achieve early detection of fake news,which is of It has great practical significance and theoretical value for the stability of the news ecology of the whole network.