| Video face deepfake detection is an emerging computer vision task aimed at detecting whether facial content in a video has been tampered with using deepfake methods.This task is of significant importance for network content security.However,video face deepfake detection also encounters many technical challenges.As video content is manipulated,it inevitably leaves behind deepfake traces in space.However,most deepfake detectors only focus on texture-level spatial forgery traces,ignoring the importance of inter-regional relationships.And the spatio-temporal deepfake traces,such as unnatural facial motion,are crucial for detection.But detectors based on spatio-temporal features are short of specific design and hardly extract spatio-temporal deepfake traces.In addition,Deepfake videos often have differential frequency domain characteristics from real videos.However,prior methods only focus on spatial frequency traces while they lack of exploring temporal frequency information.In summary,it is the key and difficult points to extract spatial,temporal and frequency traces.In view of the significant application value and technical challenges of the video face deepfake detection task,this thesis carries out a systematic study on the key technologies of video face deepfake detection.Our main contributions include:1.Deepfake detection based on spatial inconsistency measuringDeepfak e methods usually do not directly generate complete images but instead generate facial area images and blend them into real images.Therefore,there is inevitably a spatial inconsistency between the fake facial area and the real background.However,the deepfake detection methods based on spatial forgery traces often focus on local texture features and lack of the ability to measure spatial inconsistencies in the whole image,resulting in unsatisfactory detection performance.To address this problem,this thesis proposes a Pixel-Region Relationship Network that combines pixel-level and region-level relationships in image-level input to measure spatial inconsistencies.Specifically,the pixel relationship in the feature map is used as auxiliary information to enhance feature distinguishability.Then the model predicts the location of the fake region and uses attention mechanisms to extract features from fake and real regions.Finally,multiple measurement methods are combined to calculate the spatial inconsistencies in the input image.This thesis conducts extensive experiments on multiple deepfake detection datasets.The proposed method achieves 86.13%accuracy which is 3.19%higher than the baseline.And the experiments also demonstrate the superiority of the proposed model in robustness.2.Deepfake detection based on spatio-temporal inconsistency modelingMost deepfake methods first generate fake frames and then stitch them together into a complete video,ignoring the correlation between frames in the video.Therefore,deepfake videos inevitably exhibit spatial-temporal inconsistencies.However,the detection methods based on spatio-temporal features are difficult to effectively model spatio-temporal forgery traces without specific designs.To address this problem,this thesis constructs spatio-temporal graphs of deepfake videos and proposes a SpatioTemporal Graph Network to model spatio-temporal inconsistencies,to learn the global spatial inconsistency and long-distance temporal incoherence between nodes with graph convolutions.Through extensive experimental analysis,we demonstrate that the proposed model can utilize the advantages of both spatial and temporal forgery features,exhibiting excellent detection accuracy and generalization ability.The proposed method outperforms the baseline by 3.56%accuracy in NeuralTextures,and achieves 0.740 AUC in DFDC.3.Deepfake detection based on frequency domain information miningDeepfake videos and real videos often have similar content and characteristics in the color space,but they typically exhibit more significant differences in the frequency domain.Some existing works refer to traditional frequency domain processing methods to uncover spatial frequency domain forgery traces in deepfake videos with overlooking the temporal frequency domain information.To address this issue,this thesis proposes a temporal frequency domain information mining module.We combine prior knowledge of deepfake videos and data-driven training pattern,and design learnable frequency domain filters to enhance frequency components that contain forgery clues.In addition,this thesis proposes a sample generation method based on transformations in real videos to increase the diversity of frequency domain forgery traces,to help models learn the general temporal frequency domain forgery features.We conduct experiments on multiple mainstream datasets.The proposed method outperforms the baseline by 1.43%accuracy in NeuralTextures,and achieves 0.746 AUC in DFDC.In summary,this thesis analyzes the specific characteristics of deepfake facial videos and explores forgery traces in the video from multiple perspectives,including spatial,spatio-temporal,and frequency domains.The thesis conducts an in-depth study on key technologies of video face deepfake detection,which improves the accuracy,robustness,and generalization ability of deepfake detectors.The research achieves certain results in scientific research and practical applications. |
