Deepfake Detection Method Via Cross-domain Multi-scale Feature Fusion

In recent years,with the rapid development of artificial intelligence technologies represented by deep neural networks,it brings great changes to the digital media field but also poses potential threats to personal information security and other areas.In particular,the widely popular deepfake techniques have been used by criminals to generate fake facial videos and spread through illegal means,which has caused undesirable effects in society and aroused widespread concern among people.Currently,a large amount of academic research has been focused on detecting such forged face videos,but most detection methods perform poorly when dealing with compressed forged faces and have poor generalization performance facing untrained datasets due to the similarity of faces and the post-processing methods that exist in the video propagation process.In this paper,we analyze the causes of the above problems and propose two approaches based on cross-domain multi-scale feature fusion to discover more artifacts with discriminative information:(1)A progressive multi-scale deepfake detection method based on noise-aware is proposed.Firstly,the proposed progressive spatial attention module produces multi-scale features by progressively focusing on different face regions and boosting the salient regions at each stage to improve the ability of the model to capture artifacts.Secondly,it combines high-frequency information of the face with RGB information to build a two-stream detection model to cope with the possible effects of post-processing algorithms.Finally,multi-scale local similarity features and global features obtained by cross-domain multiscale features are used together to detect deepfake and improve the robustness of the model.Experiments on four open source datasets such as FaceForensics++ show better performance in terms of detection accuracy and generalization compared with current state-ofthe-art algorithms.(2)A deepfake detection method based on cross-domain multi-feature knowledge distillation is proposed.Considering the high requirements for the lightweight metrics of the model when actually deploying the application,the deepfake detection models with better performance tend to be unsuitable due to their high complexity and more operations.However,common lightweight Convolutional Neural Networks(CNNs),such as ResNet and EfficientNet,are prone to overfitting and have low detection accuracy when dealing with low-quality forged faces.Therefore,we introduce the knowledge distillation approach and propose a cross-modal method to guide the student network with the teacher network to discover the missing artifact information due to compression from the frequency and color domains for efficient low-quality forged face detection.The proposed method can improve its ability to detect low-quality forged faces while maintaining a lightweight network parameter count.Experimental results show that several lightweight CNNs trained with our method achieve different levels of performance improvements in detecting low-quality forged faces.