Research And Application Of A Deepfake Video Detection Method Based On Deep Learning Method

The term Deepfakes(deep forgery technology)was originally a combination of "deep learning" and "fake".Its content mainly refers to pictures and/or video content generated by deep learning algorithms.With the further development of GAN(generative adversarial network)-based content generation methods,today’s deepfake videos have become more and more realistic,so that the human eye is indistinguishable.Deepfake videos of spoofs of politicians are also attracting a lot of attention.In the general deepfake video detection and classification tasks,most of the research is to preprocess the video data set through the frame extraction operation,and then study the extracted pictures.Many studies choose to exploit visual artifacts in images,or missing depth information,to characterize deepfake videos.Although this method can achieve better results,it will waste a lot of sound information and timing information in the video.Therefore,in order to better utilize the sound information in video datasets,this paper proposes an adaptive multimodal deep learning method to detect deepfake videos.The principle of this method is based on the mismatch of audio information and visual information,such as inconsistency between mouth shape and speech,abnormal shaking of face or mouth,etc.From these inconsistencies,a modality discordance score(MDS score)is calculated for a video for real/fake video classification.Finally,this paper conducts relevant experiments on the DFDC data set and compares it with some current popular methods.The method proposed in this paper achieves an AUC accuracy rate of 84.4%,which is higher than some other methods,reflecting the advantages of this method.superiority.The following works are mainly carried out:(1)First,most of the traditional deepfake video detection methods do not use enough modal information in the video as a starting point,and propose a deep learning method based on audio and video multimodality to detect deep forgery.video.In the detection and classification tasks of general deepfake videos,most of the researches preprocess the video data set through the frame extraction operation,and then study the extracted pictures,using the visual artifacts remaining in the video frames.Or some missing depth information to detect fake videos.Although this method can achieve better results,it will waste the sound information in the video.Therefore,in order to better utilize the sound information in video datasets,this paper proposes a multimodal-based method to detect deepfake videos.The principle of this method is based on the mismatch of audio information and video information,such as deepfake video detection through the inconsistency of mouth shape and voice,abnormal shaking of face or mouth and other features.(2)Secondly,for the detection mechanism mentioned above,an adaptive method for prediction and judgment of deepfake video is proposed.The main method is to collect the data distribution of the dataset during the training process,record the audio-video dissonance value(MDS value)of the real video and the fake video through the training process,and obtain the real video and fake video in a dataset through this method.The dissonance value of the video,which is used to further distinguish the real video from the fake video.Compared with the traditional fixed-threshold detection and classification method,the classification criteria of the video detection method based on the adaptive method for deep forgery detection proposed in this paper can be more suitable for the real data set,and can be more reasonably obtained.The judgment basis can further improve the classification effect of real/fake videos,and at the same time,the interpretability of this method is relatively stronger and the accuracy rate is higher.(3)Finally,this paper also designs and implements a prototype system for deepfake video detection.The system aims to implement a prototype system for detecting deepfake videos based on the above methods to complete the task of deepfake video detection.For a given video,the system first goes through the pre-processing of the video preprocessing,and then trains the deep neural network model through an adaptive multimodal framework,and generates the current best model through the training and testing process.The classification and determination of a given target video,and finally determine whether a video is a deepfake video.The detection results output by the system will be displayed in the corresponding system web interface,and at the same time,the possibility of the video being a real video or a fake video will be displayed as an auxiliary reference.