Rumor Detection With Visual Data

Nowadays,people are increasingly getting used to obtaining information through various social media platforms and participating in online interactions.Meanwhile,the amount of multimedia User-Generated Content including text,pictures and videos is also growing rapidly.The Sina Weibo platform is one of the most popular online social platforms,in which hundreds of millions of users have formed a new social network by following and being followed.The new formed social network makes it easy for a person's micro-blog to be forwarded and seen by more people.Meanwhile,rumors are breaking the geographical limitations and entering the cyberspace accompanied by people's social activities on the platform.The network environment where everyone can create at any time and anywhere cause the low creation cost of rumors.The rampant rumors are also worsening the network environment,and causing chaos to individuals and even the society.However,artificially identifying a huge number of microblogs is time-consuming and labor-intensive,so how to effectively perform automatic rumor detection has its important social value.The main task of rumor detection is to determine the probability of a micro-blog being a rumor by analyzing its content.However,in the existing researches,most of the work is carried out based on the text or context of the micro-blogs,and there are few researches that use only visual data to complete the rumor detection.But both rumors and non-rumored micro-blogs often contain pictures in real life,and even some rumors can be spread only by pictures.This is because pictures have their intuitive expression ability.Therefore,this paper aims to make use of the image features to carry out the rumor image detection task on the Sina Weibo platform.The core problem of the rumor image detection task is how to make full use of the image features with only image data,so that the model can better recognize the rumor image.To solve this problem,this paper proposes the feature of "Image Self-Concept",which enables the detection model to understand the image by combining the image visual and image text concepts.In addition,this paper also analyzes the pictures from multiple angles to dig out the features that can be used for rumor detection,and adopts different fusion strategies to fuse the features,and then use a neural network to determine whether an image is a rumor.The main fields involved in feature extraction are image objects detection,optical character recognition,word embedding,and discrete cosine transform.In this paper,an experiment was took in the Sina Weibo rumor image data set,and a better rumor image detection result was obtained compared with the baseline,and the validity of the feature and model proposed in this paper was also verified from the accuracy and other indicators.