| The detection of deepfake-generated images is a prominent research topic in the field of computer vision.Achieving high accuracy in distinguishing synthetic and authentic images is crucial for the algorithm.However,with the advancement of image synthesis technology,synthetic images generated by state-of-the-art models are becoming increasingly similar to real ones,posing significant challenges to the generalization ability of detection models.To make it even worse,synthetic images shared on social media platforms often undergo various modifications like JPEG or H.264 encoding,resizing,and Gaussian blurring,resulting in visual quality degradation,loss of fine details,and alterations in image texture.Therefore,the detection model must exhibit robustness in real-world scenarios to meet these demanding requirements.In response to the above situation,this article researched Deepfake synthetic image detection algorithms.To address the poor generalization ability of current detection models,this paper proposes a Deepfake synthetic image detection algorithm based on multi-feature fusion.Ideally,real images can be divided into one class,while synthesized images can be divided into different classes due to their origins in different synthetic models.If the distance between the detected image and the real image class is greater than the set threshold,it should be classified as a synthesized image.The proposed model adopts affine transformation,sampling combination,and other methods to augment the data of the training image.Then we extract the content significance information and frequency domain difference information from the image to assist the training of the CNN classification network.Moreover,a self-attention module is adopted to fuse multiple features so that the detection model can jointly measure the distance between the true and false images based on multiple differences and eliminate the dependence on a single distinguishing feature.Even when faced with unknown types of synthesized images,it exhibits good detection accuracy and better generalization.In addition,the algorithm introduces the information bottleneck theory as an unsupervised constraint to balance the impact of prior information on model training,alleviating the overfitting of the trained model and further improving the model’s prediction accuracy.In response to the problem of reduced robustness of detection models caused by the degradation of the image’s visual quality,this paper proposes a Deepfake synthetic image detection algorithm based on content masking encoding.The algorithm reduces the dimensionality of the input image and generates residual feature maps.The generated residual feature maps contain information such as the light field,depth of field,and color of the input image,while also shielding the visual content information of the image,reducing the possibility of CNN classification networks extracting pseudo features due to visual noise interference,thereby improving the anti-noise ability of the detection model.The effectiveness of this algorithm was verified by comparing it with the state-of-the-art deep learning-based detection algorithm.The contribution of this paper can be summarized as follows:(1)A semi-supervised and scalable multi-feature fusion detection algorithm is proposed to address the generalization ability of detection models.Firstly,the algorithm processes the training data according to the rules provided by prior information and generates multiple data streams.A multi-branch self-attention CNN network synchronously extracts features from multiple data streams,which utilizes different types of prior information to regularize the network training and enables the network to extract distinguishing features from multiple aspects.Moreover,the information bottleneck theory is introduced as an unsupervised regularization for the sake of the model’s generalization ability.Secondly,unlike traditional detection models that concatenate multiple features,our proposed self-attention CNN network group can gradually achieve low-level,medium,and high-level multi-feature fusion,leading to better performance in detection accuracy.Finally,the multi-feature fusion algorithm proposed in this article can defend against advanced adversarial attacks on synthesized images by augmenting and adding new prior information while maintaining the model’s generalization ability.(2)To improve the robustness of the detection model,we adopt content masking encoding and super-resolution residual feature maps when training CNN classifiers.Specifically,the super-resolution residual feature map preserves the optical field,depth of field,color,and other information of the input image while discarding the visual content information of the image.Extensive experiments demonstrate that the design can alleviate the impact of noise features on the CNN classification network and improve the robustness of the detection model. |
