A Multi-Scale and Multi-Level Feature Fusion Approach for Deepfake Face Detection

doi:10.3969/j.issn.1671-1122.2025.09.013

Abstract

Abstract:

With the advancement of deepfake technology, current forged facial features exhibit multi-scale characteristics, and forgery artifacts persist across feature hierarchies. However, existing detection approaches generally fail to fully leverage these features. To address this issue, the paper proposed a deepfake detection method based on multi-scale and multi-level feature fusion. First, an overlapping window attention unit was integrated into Swin Transformer to extract multi-scale forgery features. Next, an innovative multi-scale feature fusion module was designed, which can fuse features of different scales extracted from various levels, thereby obtaining more expressive and robust multi-level feature representations. Finally, the effectiveness of the proposed method was validated on the FaceForensics++ (FF++) and Celeb-DF(V2) datasets.

Key words: deepfake detection, multi-scale feature fusion, deep learning, spatial domain

CLC Number:

TP309

CHEN Yonghao, CAI Manchun, ZHANG Yiwen, PENG Shufan, YAO Lifeng, ZHU Yi. A Multi-Scale and Multi-Level Feature Fusion Approach for Deepfake Face Detection[J]. Netinfo Security, 2025, 25(9): 1456-1464.

Figures/Tables 10

References 34

[1]	NGUYEN T T, NGUYEN Q V H, NGUYEN D T, et al. Deep Learning for Deepfakes Creation and Detection: A Survey[EB/OL]. (2022-08-11)[2024-09-12]. https://arxiv.org/abs/1909.11573.
[2]	CHESNEY B, CITRON D. Deep Fakes: A Looming Challenge for Privacy, Democracy, and National Security[J]. California Law Review, 2019, 107: 1753-1820.
[3]	LI Xurong, JI Shouling, WU Chunming, et al. A Survey on Deepfakes and Detection Techniques[J]. Journal of Software, 2020, 32(2): 496-518.
	李旭嵘, 纪守领, 吴春明, 等. 深度伪造与检测技术综述[J]. 软件学报, 2020, 32(2): 496-518.
[4]	TOLOSANA R, VERA-RODRIGUEZ R, FIERREZ J, et al. Deepfakes and Beyond: A Survey of Face Manipulation and Fake Detection[J]. Information Fusion, 2020, 64: 131-148.
[5]	GUO Cuicui, LUO Guibo, ZHU Yuesheng. A Detection Method for Facial Expression Reenacted Forgery in Videos[C]// SPIE. Tenth International Conference on Digital Image Processing (ICDIP 2018). San Diego: SPIE, 2018: 414-422.
[6]	KOOPMAN M, RODRIGUEZ A M, GERADTS Z. Detection of Deepfake Video Manipulation[C]// IAPR. The 20th Irish Machine Vision and Image Processing Conference (IMVIP). Heidelberg: Springer, 2018: 133-136.
[7]	MCClOSKEY S, ALBRIGHT M. Detecting GAN-Generated Imagery Using Saturation Cues[C]// IEEE. 2019 IEEE International Conference on Image Processing (ICIP). New York: IEEE, 2019: 4584-4588.
[8]	BOHAEK M, FARID H. Protecting World Leaders against Deep Fakes Using Facial, Gestural, and Vocal Mannerisms[EB/OL]. (2022-07-20)[2024-11-12]. https://www.nstl.gov.cn/paper_detail.html?id=1e2166894a6601a5f4ae6d8e93d36a18.
[9]	LI Yuezun, CHANG M, LYU Siwei. In Ictu Oculi: Exposing AI Created Fake Videos by Detecting Eye Blinking[C]// IEEE. 2018 IEEE International Workshop on Information Forensics and Security (WIFS). New York: IEEE, 2018: 1-7.
[10]	MATERN F, RIESS C, STAMMINGER M. Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations[C]// IEEE. 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW). New York: IEEE, 2019: 83-92.
[11]	YANG Xin, LI Yuezun, LYU Siwei. Exposing Deep Fakes Using Inconsistent Head Poses[C]// IEEE. 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). New York: IEEE, 2019: 8261-8265.
[12]	SUN Zekun, HAN Yujie, HUA Zeyu, et al. Improving the Efficiency and Robustness of Deepfakes Detection through Precise Geometric Features[C]// IEEE. The IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2021: 3609-3618.
[13]	PENG Shufan, CAI Manchun, MA Rui, et al. Deepfake Detection Algorithm for High-Frequency Components of Shallow Features[J]. Laser & Optoelectronics Progress, 2023, 60(10): 1-11.
	彭舒凡, 蔡满春, 马瑞, 等. 面向浅层特征高频分量的深度伪造检测算法[J]. 激光与光电子进展, 2023, 60(10): 1-11.
[14]	ZHAO Hanqing, ZHOU Wenbo, CHEN Dongdong, et al. Multi-Attentional Deepfake Detection[C]// IEEE. The IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2021: 2185-2194.
[15]	YANG Shaocong, WANG Jian, SUN Yunlian, et al. Multi-Level Features Global Consistency for Human Facial Deepfake Detection[J]. Journal of Image and Graphics, 2022, 27(9): 2708-2720.
	杨少聪, 王健, 孙运莲, 等. 多级特征全局一致性的伪造人脸检测[J]. 中国图象图形学报, 2022, 27(9): 2708-2720.
[16]	AFCHAR D, NOZICK V, YAMAGISHI J, et al. Mesonet: A Compact Facial Video Forgery Detection Network[C]// IEEE. 2018 IEEE International Workshop on Information Forensics and Security (WIFS). New York: IEEE, 2018: 1-7.
[17]	NGUYEN H H, YAMAGISHI J, ECHIZEN I. Use of a Capsule Network to Detect Fake Images and Videos[EB/OL]. (2019-09-19)[2024-09-12]. https://arxiv.org/abs/1910.12467.
[18]	AMERINI I, GALTERI L, CALDELLI R, et al. Deepfake Video Detection through Optical Flow Based Cnn[C]// IEEE. The IEEE/CVF International Conference on Computer Vision Workshops. New York: IEEE, 2019: 1-10.
[19]	SABIR E, CHENG Jiaxin, JAISWAL A, et al. Recurrent Convolutional Strategies for Face Manipulation Detection in Videos[J]. Interfaces (GUI), 2019, 3(1): 80-87.
[20]	MI Zhongjie, JIANG Xinghao, SUN Tanfeng, et al. GAN-Generated Image Detection with Self-Attention Mechanism against GAN Generator Defect[J]. IEEE Journal of Selected Topics in Signal Processing, 2020, 14(5): 969-981.
[21]	GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative Adversarial Nets[C]// NIPS.Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge: MIT Press, 2014: 2672-2680.
[22]	DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An Image Is Worth 16x16 Words: Transformers for Image Recognition at Scale[EB/OL]. (2010-11-29)[2024-09-12]. https://arxiv.org/pdf/2010.11929/1000.
[23]	WODAJO D, EMIRE S A. Haricot Beans Flour: Effect of Varieties and Processing Methods to Favor the Utilization of Underconsumed Common Beans[J]. International Journal of Food Properties, 2022, 25(1): 1186-1202.
[24]	DOLHANSKY B, BITTON J, PFLAUM B, et al. The Deepfake Detection Challenge (DFDC) Dataset[EB/OL]. (2020-10-28)[2024-09-12]. https://arxiv.org/abs/2006.07397.
[25]	COCCOMINI D A, MESSINA N, GENNARO C, et al. Combining Efficientnet and Vision Transformers for Video Deepfake Detection[C]// Springer. Image Analysis and Processing-ICIAP 2022: 21st International Conference. Heidelberg: Springer, 2022: 219-229.
[26]	CHEN Xiangyu, WANG Xintao, ZHOU Jiantao, et al. Activating More Pixels in Image Super-Resolution Transformer[C]// IEEE. The IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2023: 22367-22377.
[27]	LIU Ze, LIN Yutong, CAO Yue, et al. Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows[C]// IEEE. The IEEE/CVF International Conference on Computer Vision. New York: IEEE, 2021: 10012-10022.
[28]	ZHANG Kaipeng, ZHANG Zhanpeng, LI Zhifeng, et al. Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks[J]. IEEE Signal Processing Letters, 2016, 23(10): 1499-1503.
[29]	LIU Yichao, SHAO Zongru, HOFFMANN N. Global Attention Mechanism: Retain Information to Enhance Channel-Spatial Interactions[EB/OL]. (2021-06-21)[2024-09-12]. https://arxiv.org/abs/2112.05561.
[30]	ROSSLER A, COZZOLINO D, VERDOLIVA L, et al. Faceforensics++: Learning to Detect Manipulated Facial Images[C]// IEEE. The IEEE/CVF International Conference on Computer Vision. New York: IEEE, 2019: 1-11.
[31]	LI Yuezun, YANG Xin, SUN Pu, et al. Celeb-Df: A Large-Scale Challenging Dataset for Deepfake Forensics[C]// IEEE. The IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2020: 3207-3216.
[32]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep Residual Learning for Image Recognition[EB/OL]. (2016-08-16)[2024-09-12]. https://arxiv.org/abs/1512.03385.
[33]	WANG Junke, WU Zuxuan, OUYANG Wenhao, et al. M2tr: Multi-Modal Multi-Scale Transformers for Deepfake Detection[C]// ACM. The 2022 International Conference on Multimedia Retrieval. New York: ACM, 2022: 615-623.
[34]	QIAN Yuyang, YIN Guojun, SHENG Lu, et al. Thinking in Frequency: Face Forgery Detection by Mining Frequency-Aware Clues[C]// Springer. Computer Vision-ECCV 2020: 16th European Conference. Heidelberg: Springer, 2020: 86-103.

类别	配置
CPU	i9-12900H
显卡	GeForce RTX 3090
内存大小	32 GB
操作系统	Ubuntu16.04 64 bit
编程语言	Python3.9.15
深度学习框架	Pytorch1.13.1
并行计算架构	CUDA 11.8
深度神经网络库	cuDNN 8.6.0

方法	FF++ (C23)		Celeb-DF(V2)		Params/个	计算量/GFLOPs
方法	ACC	AUC	ACC	AUC	Params/个	计算量/GFLOPs
MesoNet	85.9%	88.0%	—	—	—	—
Xception	95.3%	96.3%	—	97.6%	2.58×10⁷	18.9
ResNet	93.2%	95.4%	96.9%	98.6%	4.46×10⁷	23.6
F3-Net	97.5%	98.1%	—	—	—	67.3
Cross Vit Net	98.1%	98.6%	98.4%	99.0%	1.01×10⁸	—
M2TR	97.9%	99.5%	98.3%	99.4%	—	—
本文方法	98.8%	99.7%	99.3%	99.4%	7.13×10⁷	38.4

方法	Deepfakes （C23-C40）		FaceSwap （C23-C40）
方法	ACC	AUC	ACC	AUC
MesoNet	84.4%	84.8%	84.1%	85.5%
Xception	86.0%	87.8%	92.0%	92.7%
ResNet	85.5%	86.3%	83.5%	87.6%
Cross Vit EfficientNet	92.8%	94.6%	92.1%	92.3%
本文方法	94.5%	95.5%	93.7%	95.2%

方法	AUC
LRNet	56.9%
MesoNet	53.9%
Xception	66.8%
Cross Vit EfficientNet	67.2%
本文方法	72.1%

方法	AUC
方法	Deepfakes(C23)	FaceSwap(C23)	Celeb-DF(V2)
Swin（基线模型）	97.9%	98.5%	97.9%
Swin+OCAB(SwinOB)	99.2%	99.0%	98.6%
Swin+OCAB+MFFM	99.7%	99.7%	99.4%