基于图像细粒度特征的深度伪造检测算法

doi:10.3969/j.issn.1671-1122.2022.11.010

信息网络安全 ›› 2022, Vol. 22 ›› Issue (11): 77-84.doi: 10.3969/j.issn.1671-1122.2022.11.010

基于图像细粒度特征的深度伪造检测算法

彭舒凡, 蔡满春(), 刘晓文, 马瑞

中国人民公安大学信息网络安全学院，北京 100038

收稿日期:2022-06-30 出版日期:2022-11-10 发布日期:2022-11-16
通讯作者: 蔡满春 E-mail:caimanchun@ppsuc.edu.cn
作者简介:彭舒凡（1998—），男，江苏，硕士研究生，主要研究方向为信息网络安全|蔡满春（1972—），男，河北，副教授，博士，主要研究方向为密码学与通信保密|刘晓文（1997—），男，山东，硕士研究生，主要研究方向为遥感图像融合|马瑞（1997—），男，江苏，硕士研究生，主要研究方向为深度伪造检测
基金资助:
“十三五”国家密码发展基金密码理论研究重点课题(MMJJ20180108);中国人民公安大学2020年基本科研业务费重大项目(2020JKF101)

Deepfake Detection Algorithm Based on Image Fine-Grained Features

PENG Shufan, CAI Manchun(), LIU Xiaowen, MA Rui

College of Information Network Security, People’s Public Security University of China, Beijing 100038, China

Received:2022-06-30 Online:2022-11-10 Published:2022-11-16
Contact: CAI Manchun E-mail:caimanchun@ppsuc.edu.cn

摘要/Abstract

摘要：

随着深度学习的发展，深度伪造生成模型解决了生成图像中存在明显伪影的问题，但深度伪造图像与真实图像之间的区别往往是细微且局部的。因此，文章构建了一个基于图像细粒度特征的检测模型FGDD。针对深度伪造检测任务中仅使用粗粒度特征的不足，FGDD利用多分支模型充分学习样本图像的细粒度特征，通过引入通道注意力机制以及优化激活图掩膜定位策略提升面部敏感区域定位的精度。在激活图中，利用多级滑动窗口提取样本中的高区分度细微特征，通过AugMix数据增强策略提升模型对于细粒度特征的鲁棒性。实验结果表明，FGDD在多个数据集上的测试准确率优于主流算法，证明了基于图像细粒度特征的深度伪造检测算法的有效性。

关键词: 深度伪造, 深度伪造检测, 细粒度特征, 数据增强

Abstract:

With the development of deep learning, deepfake generation models have overcome the drawback of having obvious artifacts in the generated images, but the difference between deepfake images and real images is often subtle and partial. Therefore, a detection model FGDD based on fine-grained features of images was constructed in this paper. To address the shortcomings of using only coarse-grained features, FGDD fully learned the fine-grained features of the sample images by multi-branch, and improved the accuracy of locating sensitive facial regions by introducing a channel attention mechanism and an optimized activation map mask localization strategy. In the activation graph, the multi-level sliding windows were used to extract the highly differentiated subtle features in the samples, and the robustness of the model for fine-grained features was improved by using AugMix data enhancement strategy. The experimental results show that the tested accuracy of FGDD on several datasets outperform the mainstream algorithms, proving the effectiveness of the detection method based on fine-grained features of images.

Key words: deepfake, deepfake detection, fine-grained features, data augment

中图分类号:

TP309

彭舒凡, 蔡满春, 刘晓文, 马瑞. 基于图像细粒度特征的深度伪造检测算法[J]. 信息网络安全, 2022, 22(11): 77-84.

PENG Shufan, CAI Manchun, LIU Xiaowen, MA Rui. Deepfake Detection Algorithm Based on Image Fine-Grained Features[J]. Netinfo Security, 2022, 22(11): 77-84.

图/表 12

图1

图2

图3

表1

图4

表2

图5

图6

图7

表3

表4

表5

参考文献 20

[1]	GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative Adversarial Nets[J]. Advances in Neural Information Processing Systems, 2014(2): 2672-2680.
[2]	JUEFEI-XU F, WANG Run, HUANG Yihao, et al. Countering Malicious Deepfakes: Survey, Battleground, and Horizon[J]. International Journal of Computer Vision, 2022(130): 1678-1734.
[3]	WODAJO D, ATNAFU S. Deepfake Video Detection Using Convolutional Vision Transformer[EB/OL]. (2021-03-11)[2022-04-01]. .
[4]	AFCHAR D, NOZICK V, YAMAGISHI J, et al. MesoNet: A Compact Facial Video Forgery Detection Network[C]// IEEE. 2018 IEEE International Workshop on Information Forensics and Security (WIFS). New York:IEEE, 2018: 1-7.
[5]	BAYAR B, STAMM M C. A Deep Learning Approach to Universal Image Manipulation Detection Using A New Convolutional Layer[C]// ACM. Proceedings of the 4th ACM Workshop on Information Hiding and Multimedia Security. New York: ACM, 2016: 5-10.
[6]	NGUYEN H H, YAMAGISHI J, ECHIZEN I. Capsule-Forensics: Using Capsule Networks to Detect Forged Images and Videos[C]// IEEE. 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). New York:IEEE, 2019: 2307-2311.
[7]	MASI I, KILLEKAR A, MASCARENHAS R M, et al. Two-Branch Recurrent Network for Isolating Deepfakes in Videos[C]//Springer. European Conference on Computer Vision. Berlin:Springer, 2020: 667-684.
[8]	WANG Xin, GUO Hui, HU Shu, et al. GAN-Generated Faces Detection: A Survey and New Perspectives[EB/OL]. (2022-03-16)[2022-04-01]. .
[9]	HU Shu, LIYuezun, LYU Siwei. Exposing GAN-Generated Faces Using Inconsistent Corneal Specular Highlights[C]// IEEE. 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). New York:IEEE, 2021: 2500-2504.
[10]	MATERN F, RIESS C, STAMMINGER M. Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations[C]// IEEE. 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW). New York:IEEE, 2019: 83-92.
[11]	GUO Hui, HU Shu, WANG Xin, et al. Eyes Tell All: Irregular Pupil Shapes Reveal GAN-Generated Faces[EB/OL]. (2022-04-27)[2022-04-29]. .
[12]	GUARNERA L, GIUDICE O, BATTIATO S. Deepfake Detection by Analyzing Convolutional Traces[C]// IEEE. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. New York: IEEE, 2020: 666-667.
[13]	WANG Shengyu, WANG O, ZHANG R, et al. CNN-Generated Images are Surprisingly Easy to Spot… for Now[C]// IEEE. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2020: 8695-8704.
[14]	LI Yuezun, LYU Siwei. Exposing Deepfake Videos by Detecting Face Warping Artifacts[EB/OL]. (2018-11-01)[2022-02-15]. .
[15]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1904-1916. doi: 10.1109/TPAMI.2015.2389824 pmid: 26353135
[16]	ZHANG Fan, LI Meng, ZHAI Guisheng, et al. Multi-Branch and Multi-Scale Attention Learning for Fine-Grained Visual Categorization[C]// Springer. International Conference on Multimedia Modeling. Berlin:Springer, 2021: 136-147.
[17]	ROSSLER A, COZZOLINO D, VERDOLIVA L, et al. FaceForensics++: Learning to Detect Manipulated Facial Images[C]// IEEE. Proceedings of the IEEE/CVF International Conference on Computer Vision. New York: IEEE, 2019: 1-11.
[18]	LI Yuezun, YANG Xin, SUN Pu, et al. Celeb-DF: A Large-Scale Challenging Dataset for Deepfake Forensics[C]// IEEE. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2020: 3207-3216.
[19]	WANG Qilong, WU Banggu, ZHU Pengfei, et al. ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks[C]// IEEE. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2020: 11531-11539.
[20]	HENDRYCKS D, MU N, CUBUK E D, et al. AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty[EB/OL]. (2019-12-05)[2022-02-01]. .

数据集	Deepfakes	FaceSwap	Celeb-DF-v2
训练集	41936 images	41945 images	143192 images
测试集	17089 images/600 videos	15600 images/600 videos	41618 images/1354 videos

$Mas{{k}_{\text{Re}}}$	Deepfakes		FaceSwap		Celeb-DF-v2
$Mas{{k}_{\text{Re}}}$	准确率	召回率	准确率	召回率	准确率	召回率
$Mc{{c}_{\text{LB}}}\bigcap Mc{{c}_{\text{Final }}}$	98.83%	98.67%	99.17%	99.33%	98.30%	97.73%
$Mc{{c}_{\text{Final }}}$	99.17%	99.33%	99.50%	99.67%	98.52%	98.86%
$Mas{{k}_{\text{LB}}}\bigcap Mc{{c}_{\text{Final }}}$	99.67%	99.67%	99.83%	100.00%	98.97%	99.43%

不使用数据增强的情况				Deepfakes		FaceSwap		Celeb-DF-v2
+ECA	+${{L}_{\text{raw }}}$	+${{L}_{\text{local }}}$	+${{L}_{\text{fine}}}$	准确率	召回率	准确率	召回率	准确率	召回率
√	√	—	—	98.17%	97.67%	98.83%	98.67%	97.12%	97.16%
√	√	√	—	98.83%	99.00%	99.00%	99.33%	97.42%	97.73%
√	√	—	√	99.00%	99.00%	99.33%	99.67%	98.30%	98.30%
—	√	√	√	99.33%	99.33%	99.83%	100.00%	98.52%	98.86%
√	√	√	√	99.67%	99.67%	99.83%	100.00%	98.97%	99.43%

模型	Deepfakes		FaceSwap		Celeb-DF-v2
模型	准确率	召回率	准确率	召回率	准确率	召回率
EfficientNet-B4	98.33%	98.00%	98.83%	99.00%	97.42%	97.73%
XceptionNet	97.83%	97.67%	98.17%	98.00%	96.82%	96.59%
MesoNet	95.50%	96.00%	93.33%	93.33%	91.21%	90.91%
ResNet50	93.50%	93.67%	94.83%	94.67%	93.87%	94.32%
Nguyen	94.50%	94.33%	93.67%	93.00%	92.76%	92.61%
FWA	94.33%	94.67%	93.50%	93.33%	92.47%	93.18%
DSP-FWA	98.17%	98.33%	97.83%	98.00%	96.90%	97.16%
FGDD	99.67%	99.67%	99.83%	100.00%	98.97%	99.43%

模型	参数	每秒浮点运算
MesoNet(MesoInception4)	0.04 M	1.85 G
VGG19	20.62 M	40.06 G
EfficientNet-B4	19.53 M	6.26 G
ResNet50	25.76 M	17.06 G
XceptionNet	22.97 M	18.62 G
DSP-FWA(ResNet101-based)	42.65 M	16.90 G
FGDD	22.97 M	37.24 G

基于图像细粒度特征的深度伪造检测算法

Deepfake Detection Algorithm Based on Image Fine-Grained Features

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 12

参考文献 20

相关文章 3

编辑推荐

Metrics

本文评价

[1]	胡勉宁, 李欣, 李明锋, 孙海春. 面向诈骗短信息识别的融合多策略数据增强技术研究[J]. 信息网络安全, 2022, 22(10): 121-128.
[2]	朱新同, 唐云祁, 耿鹏志. 基于特征融合的篡改与深度伪造图像检测算法[J]. 信息网络安全, 2021, 21(8): 70-81.
[3]	马瑞, 蔡满春, 彭舒凡. 一种基于改进的Xception网络的深度伪造视频检测模型[J]. 信息网络安全, 2021, 21(12): 109-117.