Research On Rumor Detection Method Based On Privacy Protection And Multi-modal

With the rapid development of social media,all kinds of information are published and repeated all the time.While people enjoy the convenience of the information age,they are also surrounded by many rumors and false information.The widespread spread of rumors is easy to pose a serious threat to network security and social stability.How to effectively identify rumors from the extensive and multimodal information of social media has become a research hotspot.In the detection research,we need to train the algorithm model based on a large number of data sets,which is easy to cause the disclosure of private data sets or private data,and damage the interests of social media platforms or users.In this thesis,based on deep learning method,text,visual and user modes are effectively integrated to improve the performance of social media rumor detection;differential privacy protection mechanism is integrated into the model,and the effectiveness of privacy protection mechanism is verified from two aspects of theory and inversion attack.Firstly,text sentence representation,text word representation,visual representation and user representation are extracted by using BERT,Glo Ve,VGG-19 and Z-score standardization.Considering that it is difficult to distinguish rumors only from single text or image,text and vision are matched at the representation level,and attention mechanism is fused to generate word representation and visual representation.The attention mechanism is also integrated into the early fusion and late fusion,so that the model can automatically consider the proportion of each modal in the model.The Dempster combination rule is used to combine the early fusion and late fusion,and a multi-modal fusion method based on hybrid fusion(DHF)is designed.The results show that the accuracy and F1 value are higher on Twitter and microblog,which is 15%-20% higher than the current advanced multimodal rumor detection method.Secondly,in order to protect the training set which involves a large amount of user information from malicious acquisition by illegal elements in the process of model training,we design two privacy protection methods based on differential privacy mechanism,which are adding Gaussian noise to the gradient and middle layer of deep learning network training process,and analyzes the impact of noise on the utility of the model and the reasons.Then we analyze the effectiveness of the privacy protection method based on differential privacy theory.Finally,to further verify the effectiveness of the differential privacy mechanism for privacy protection,we make inversion attack on the multi-modal rumor detection model.The training set is inverted based on gradient optimization method,and the restoration degree of inversion attack on the training set is compared with whether there is privacy protection mechanism in the model,so as to verify the effectiveness of privacy protection mechanism.To solve the problem that the derivative of the model cannot be calculated under the condition of black box,so the gradient optimization method cannot be used.In this thesis,we train the shadow model close to the original model as the basis of black box attack,and realize the inversion attack under the two environments of white box and black box.