Research On Deepfake Image Detection Based On Deep Learning To Integrate Multi-dimensional Recognition Features

Deepfake uses deep learning algorithms to conduct "game-style" confrontation training on a large number of samples,replace pictures or videos with original audio-visual materials,combine and replace individual voices,facial expressions,and body movements,correct background tone and blur boundaries to achieve the effect of "fake".The technology can reproduce personal biometric characteristics,mainly divided into identity replacement and retention.If the technology is misused,it will become a crime tool and violate people’s legitimate rights and interests.A complete system has yet to be established in deepfake image detection technology research in forensic science identification.Although the detection features have changed from artificial tamper feature to biological signal feature and image attribute feature,and the feature type has developed from image convolution feature to statistical feature,the current mainstream detection technology is still dominated by image convolution feature and lacks the application of quantitative feature.The model interpretability is low.However,the image has an ample storage space,and the convolutional neural network has a deep layer,leading to an ample space,long time,and high operation cost.Therefore,this paper carries out the research of deep learning fusion of multi-dimensional quantization features to detect deepfake images.Compared with image convolution features,quantization features have strong explanatory properties and higher utilization values in the identification field.This experimental study is divided into three parts:In the first part,it is image quantization feature screening.Based on the Forgery Net dataset,the images were divided into four groups for exploration:(1)Real image and deepfake image;(2)Deepfake image of identity replacement and deepfake image of identity retention;(3)Real image and Deepfake image of identity replacement;(4)Real image and deepfake image of identity retention.This paper proposes two feature selection methods: the method based on mathematical statistics and the method based on the mainstream deepfake technology imaging rule.The number of features screened by the two feature selection methods is different.Among them,the number of features screened by the mathematical statistics method is more.In contrast,the number of features screened by the mainstream deepfake technology imaging rule method is less,which is lower than the number of features screened by PCA,whose cumulative contribution rate is 95%.This provides a new idea for the index screening of deepfake image inspection and identification.In the second part,deepfake image detection is based on the quantization feature of screened images.In the experimental research,the first is to use the quantitative features obtained by screening through XGBoost,logistic regression classifier,linear support vector machine,multilayer perceptron and Tab Net to detect the deepfake image and make a comparison;Second,the six mainstream convolutional neural networks Res Net50V2,Xception Net,Efficient Net V2 S,Efficient Net B4,Dense Net121 and Mobile Net V2 are used to detect the deepfake images and compare their accuracy with the former ones.The test results show that the detection accuracy based on quantized data is higher than that based on a convolutional neural network.The model training and detection time based on quantitative data is lower than that based on a convolutional neural network.At the same time,the image features screened by the proposed method are of great value in deepfake image inspection.In the third part,a new deep learning detection network is built according to the characteristics of deepfake images.In this experiment,Tab Net and Mobile Net V2 networks,screening features and the probabilistic features of binary results of convolutional neural networks were integrated.At the same time,the new model was optimized and parameters analyzed.Finally,ROC,AUC and Acc were used to evaluate the new model.The importance scores of each feature in network training and testing and the interaction degree graphs of some samples and features were output to improve the interpretability of the model further and reflect the overall performance of the model.The experimental results show that the adaptive performance and accuracy of the model were improved by integrating multi-dimensional features.To sum up,two detection methods,mathematical statistics and imaging rules of deepfake images,are proposed in this paper to conduct multivariate statistics and analysis of gray cooccurrence matrix features and color moment features in the color space component of the image and select features with high correlation and stability with deepfake images for algorithm detection.In addition,the convolutional neural network Mobile Net V2 and Tab Net network based on tab-based deep learning of tabular data are integrated to detect deepfake images,and the probabilistic features of convolutional neural network results and the quantitative features of screening are integrated to improve further the detection accuracy and model performance of the network.It is proved that it is feasible to detect deepfake images based on the imaging rules of mathematical statistics and deepfake technology,which improves the interpretability of the detection model,provides a new idea and foundation for the selection of deepfake image identification indicators,and makes up the gap in deepfake inspection based on quantitative features.