Deepfake Detection Algorithm Based on Image Fine-Grained Features

doi:10.3969/j.issn.1671-1122.2022.11.010

Abstract

Abstract:

With the development of deep learning, deepfake generation models have overcome the drawback of having obvious artifacts in the generated images, but the difference between deepfake images and real images is often subtle and partial. Therefore, a detection model FGDD based on fine-grained features of images was constructed in this paper. To address the shortcomings of using only coarse-grained features, FGDD fully learned the fine-grained features of the sample images by multi-branch, and improved the accuracy of locating sensitive facial regions by introducing a channel attention mechanism and an optimized activation map mask localization strategy. In the activation graph, the multi-level sliding windows were used to extract the highly differentiated subtle features in the samples, and the robustness of the model for fine-grained features was improved by using AugMix data enhancement strategy. The experimental results show that the tested accuracy of FGDD on several datasets outperform the mainstream algorithms, proving the effectiveness of the detection method based on fine-grained features of images.

Key words: deepfake, deepfake detection, fine-grained features, data augment

CLC Number:

TP309

PENG Shufan, CAI Manchun, LIU Xiaowen, MA Rui. Deepfake Detection Algorithm Based on Image Fine-Grained Features[J]. Netinfo Security, 2022, 22(11): 77-84.

Figures/Tables 12

References 20

[1]	GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative Adversarial Nets[J]. Advances in Neural Information Processing Systems, 2014(2): 2672-2680.
[2]	JUEFEI-XU F, WANG Run, HUANG Yihao, et al. Countering Malicious Deepfakes: Survey, Battleground, and Horizon[J]. International Journal of Computer Vision, 2022(130): 1678-1734.
[3]	WODAJO D, ATNAFU S. Deepfake Video Detection Using Convolutional Vision Transformer[EB/OL]. (2021-03-11)[2022-04-01]. .
[4]	AFCHAR D, NOZICK V, YAMAGISHI J, et al. MesoNet: A Compact Facial Video Forgery Detection Network[C]// IEEE. 2018 IEEE International Workshop on Information Forensics and Security (WIFS). New York:IEEE, 2018: 1-7.
[5]	BAYAR B, STAMM M C. A Deep Learning Approach to Universal Image Manipulation Detection Using A New Convolutional Layer[C]// ACM. Proceedings of the 4th ACM Workshop on Information Hiding and Multimedia Security. New York: ACM, 2016: 5-10.
[6]	NGUYEN H H, YAMAGISHI J, ECHIZEN I. Capsule-Forensics: Using Capsule Networks to Detect Forged Images and Videos[C]// IEEE. 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). New York:IEEE, 2019: 2307-2311.
[7]	MASI I, KILLEKAR A, MASCARENHAS R M, et al. Two-Branch Recurrent Network for Isolating Deepfakes in Videos[C]//Springer. European Conference on Computer Vision. Berlin:Springer, 2020: 667-684.
[8]	WANG Xin, GUO Hui, HU Shu, et al. GAN-Generated Faces Detection: A Survey and New Perspectives[EB/OL]. (2022-03-16)[2022-04-01]. .
[9]	HU Shu, LIYuezun, LYU Siwei. Exposing GAN-Generated Faces Using Inconsistent Corneal Specular Highlights[C]// IEEE. 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). New York:IEEE, 2021: 2500-2504.
[10]	MATERN F, RIESS C, STAMMINGER M. Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations[C]// IEEE. 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW). New York:IEEE, 2019: 83-92.
[11]	GUO Hui, HU Shu, WANG Xin, et al. Eyes Tell All: Irregular Pupil Shapes Reveal GAN-Generated Faces[EB/OL]. (2022-04-27)[2022-04-29]. .
[12]	GUARNERA L, GIUDICE O, BATTIATO S. Deepfake Detection by Analyzing Convolutional Traces[C]// IEEE. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. New York: IEEE, 2020: 666-667.
[13]	WANG Shengyu, WANG O, ZHANG R, et al. CNN-Generated Images are Surprisingly Easy to Spot… for Now[C]// IEEE. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2020: 8695-8704.
[14]	LI Yuezun, LYU Siwei. Exposing Deepfake Videos by Detecting Face Warping Artifacts[EB/OL]. (2018-11-01)[2022-02-15]. .
[15]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1904-1916. doi: 10.1109/TPAMI.2015.2389824 pmid: 26353135
[16]	ZHANG Fan, LI Meng, ZHAI Guisheng, et al. Multi-Branch and Multi-Scale Attention Learning for Fine-Grained Visual Categorization[C]// Springer. International Conference on Multimedia Modeling. Berlin:Springer, 2021: 136-147.
[17]	ROSSLER A, COZZOLINO D, VERDOLIVA L, et al. FaceForensics++: Learning to Detect Manipulated Facial Images[C]// IEEE. Proceedings of the IEEE/CVF International Conference on Computer Vision. New York: IEEE, 2019: 1-11.
[18]	LI Yuezun, YANG Xin, SUN Pu, et al. Celeb-DF: A Large-Scale Challenging Dataset for Deepfake Forensics[C]// IEEE. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2020: 3207-3216.
[19]	WANG Qilong, WU Banggu, ZHU Pengfei, et al. ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks[C]// IEEE. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2020: 11531-11539.
[20]	HENDRYCKS D, MU N, CUBUK E D, et al. AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty[EB/OL]. (2019-12-05)[2022-02-01]. .

数据集	Deepfakes	FaceSwap	Celeb-DF-v2
训练集	41936 images	41945 images	143192 images
测试集	17089 images/600 videos	15600 images/600 videos	41618 images/1354 videos

$Mas{{k}_{\text{Re}}}$	Deepfakes		FaceSwap		Celeb-DF-v2
$Mas{{k}_{\text{Re}}}$	准确率	召回率	准确率	召回率	准确率	召回率
$Mc{{c}_{\text{LB}}}\bigcap Mc{{c}_{\text{Final }}}$	98.83%	98.67%	99.17%	99.33%	98.30%	97.73%
$Mc{{c}_{\text{Final }}}$	99.17%	99.33%	99.50%	99.67%	98.52%	98.86%
$Mas{{k}_{\text{LB}}}\bigcap Mc{{c}_{\text{Final }}}$	99.67%	99.67%	99.83%	100.00%	98.97%	99.43%

不使用数据增强的情况				Deepfakes		FaceSwap		Celeb-DF-v2
+ECA	+${{L}_{\text{raw }}}$	+${{L}_{\text{local }}}$	+${{L}_{\text{fine}}}$	准确率	召回率	准确率	召回率	准确率	召回率
√	√	—	—	98.17%	97.67%	98.83%	98.67%	97.12%	97.16%
√	√	√	—	98.83%	99.00%	99.00%	99.33%	97.42%	97.73%
√	√	—	√	99.00%	99.00%	99.33%	99.67%	98.30%	98.30%
—	√	√	√	99.33%	99.33%	99.83%	100.00%	98.52%	98.86%
√	√	√	√	99.67%	99.67%	99.83%	100.00%	98.97%	99.43%

模型	Deepfakes		FaceSwap		Celeb-DF-v2
模型	准确率	召回率	准确率	召回率	准确率	召回率
EfficientNet-B4	98.33%	98.00%	98.83%	99.00%	97.42%	97.73%
XceptionNet	97.83%	97.67%	98.17%	98.00%	96.82%	96.59%
MesoNet	95.50%	96.00%	93.33%	93.33%	91.21%	90.91%
ResNet50	93.50%	93.67%	94.83%	94.67%	93.87%	94.32%
Nguyen	94.50%	94.33%	93.67%	93.00%	92.76%	92.61%
FWA	94.33%	94.67%	93.50%	93.33%	92.47%	93.18%
DSP-FWA	98.17%	98.33%	97.83%	98.00%	96.90%	97.16%
FGDD	99.67%	99.67%	99.83%	100.00%	98.97%	99.43%

模型	参数	每秒浮点运算
MesoNet(MesoInception4)	0.04 M	1.85 G
VGG19	20.62 M	40.06 G
EfficientNet-B4	19.53 M	6.26 G
ResNet50	25.76 M	17.06 G
XceptionNet	22.97 M	18.62 G
DSP-FWA(ResNet101-based)	42.65 M	16.90 G
FGDD	22.97 M	37.24 G