Research And Implementation Of False Information Detection Technology Based On Multi-modal Data Fusion

The development of social media has drastically reduced the cost of dissemination of information content,which has made it a medium for the dissemination of false information.Since most false information is closely related to people’s lives,its widespread dissemination has brought serious negative impacts on society.In order to enhance the credibility and attention of the content,disseminators of false information often generate false information from multiple modals.Most of the existing false information detection methods focus on the correlation between textual and visual modalities,while ignoring the important information contained in the structural characteristics of social networks.Some methods utilize social graphs to detect false information,but fail to consider that there is a serious sparsity problem in the constructed social graph,which weakens the role of social graph features.In response to the above challenges,this paper proposes two disinformation detection methods based on multimodal data fusion.The first method mainly focuses on multimodal fusion and alignment.This method jointly extracts textual modality visual modality and social graph features,using cross-modal joint attention mechanism for modal fusion,and using unsupervised loss function for modal alignment,making the content representation of posts and their representation as nodes in social graphs closer,enriches the feature representation of post from multiple angle,and mines more relevant information of posts.The second method mainly focuses on the learning of latent links in the social graph for multimodal disinformation detection,thereby alleviating the severe sparsity problem in social graph.This method calculates node similarity by combining semantic and structural features,constructs static potential links,and learns dynamic attention coefficients between nodes at the same time,so as to dynamically adjust the link relationship during model training and enrich the feature representation of post nodes.Finally,the method enhance the features of post after multi-modal fusion.The experimental results show that the above two methods have achieved significant performance improvement.Finally,this paper designs and implements a false information detection system based on multimodal data fusion,which integrates the above two algorithms in the same framework.Users can perform data processing,model training,and model testing functions through the localside visual interface,and can adjust parameters through real-time feedback results from the server side,thereby improving the false information detection performance of the model.