Research On Fake Face Detection Algorithm Based On Spatiotemporal Features

Deep learning not only brings innovation to computer vision,but also poses a threat to social security.Especially in recent years,the deepfake videos produced by faceswap technologies not only infringes personal privacy,but also affects social security.At present,a large number of researches focus on the detection of deepfake videos.However,in the design stage of detection model,there is a lack of consideration for the particularity on deepfake videos such as dynamic flaws and texture flaws,which makes it difficult to effectively integrate the spatial and temporal features of human faces.In addition,the lack of sufficient constraints will lead to the model learning redundant information,which reduces the accuracy of feature representation on forgery detection task.In view of this,this paper proposes the following two schemes for high-precision and high generalization deepfake detection(1)Based on NASNetMobile backbone model,aiming at the defects of the existing cnn-lstm based detection method,this paper proposes a spatio-temporal oriented multi task metric learning detection method.Firstly,the tampered region prediction task is introduced into the optimization target of the backbone model to form multi task learning with classification detection.At the same time,the contrast loss is introduced to ensure the separability of real and faces.At last,the long-term and short-term memory network is used as the time-domain feature extractor to achieve high-precision forged face detection.(2)Aiming at the problem that deepfake detection model inevitably learns semantic information,this paper designs a detection method based on spatiotemporal fusion and introduce digital forensics domain knowledge.Firstly,learnable filtering and spatial rich model filtering are used to extract noise features and suppress image semantic content.Then,the local encoder network and bidirectional long short memory model are used to extract the self consistent features of an image.Finally,additional time-domain features are introduced to assist the detection.The two schemes proposed in this paper have been fully tested and analyzed on a variety of most advanced large-scale datasets: Face Forensics + +,Celeb DF,Deeperforensics and DFDC-P.The methods achieve good average accuracy,and the effectiveness of different components has been proved by ablation experiments.The results show that the proposed method has a positive effect on deepfakes detection,the detection accuracy exceeds many existing optimal schemes,and the cross-data detection also achieves good results.