Deep Forgery Detection Based On Global And Local Feature Enhancement

The rapid development of deep learning and generative modeling have promoted the progress of face manipulation and forgery technologies(i.e.,deepfake),which largely lower the bar of creating photo-realistic facial images/videos.While deep forgery technology creates a variety of applications,the abuse of technology also brings many negative effects.The development of effective deep forgery detection methods has become one of the research hotspots in the computer vision community.The existing deepfake detection methods are mainly based on supervised learning to train a binary-classification model,but such methods are easy to overfit to the training set and cannot be generalized to the real usage scenarios,and the model is very sensitive to the quality of the test samples so that a slight change in the quality of the test images will cause the performance of the model to drop sharply.In order to solve the above problems,we propose to improve the model's generalization ability to unknown forgery types and image quality from the perspective of global and local feature enhancement.We achieve model's local feature enhancement through adversarial training.Adversarial training generates adversarial examples that are more difficult for the classification model and uses these samples to train detection model,prompting the model to learn more essential classification features.Aiming at the problem of poor robustness of the forgery detection model to changes in image quality,we proposes adversarial Gaussian blur,which uses pixel-wise Gaussian blur to generate adversarial examples.In addition,powerful adversarial examples are normally designed with a multi-step scheme,therefore the computational cost increases as the the number of steps increases.To control the computational cost of the multi-step scheme,we proposes to use generators to generate adversarial examples.Furthermore,by combining the generator-based adversarial Gaussian blur with other adversarial training methods,the generalization performance of the model can be further improved.Frequency domain information is an important supplement to the image.We introduces frequency domain information to achieve detection model's global feature enhancement.Forged images and real images have obvious differences in the highfrequency part,and different types of forgery generally modify different frequencies,so frequency-aware image decomposition is a means to effectively use frequency information.In addition,there are differences between the global frequency representation and the local frequency representation of images.Based on these observations,we designs two methods of global frequency decomposition and local frequency decomposition to effectively use frequency information.Both are based on the idea of frequency-aware image decomposition.The difference is that the global decomposition transforms the image as a whole to frequency domain and the local decomposition performs a sliding window frequency conversion on the image.Experiments show that the two kinds of frequency information are obviously complementary,which inspired us to design a model to use two kinds of frequency information to detect forged images.Finally,we proposes a feature enhancement model that uses both spatial image and frequency image.The model performs the fusion of frequency domain and spatial image features at each layer of the network.A large number of experiments show that both adversarial training and frequency information effectively improve the generalization ability of the model.The feature enhancement model trained using adversarial Gaussian blur based on generators achieves leading performance on a variety of commonly used deep forgery datasets,and its generalization ability is significantly better than several other recent methods.