基于共性伪造线索感知的物理和数字人脸攻击联合检测方法

doi:10.3969/j.issn.1671-1122.2025.10.011

摘要/Abstract

摘要：

人脸识别系统在实际应用中面临物理攻击与数字攻击的双重威胁。由于两类攻击存在显著异质性，通常需要依赖不同的模型分别应对。为了节约计算资源及硬件部署成本，针对物理攻击和数字攻击在特征空间中呈现显著分布差异且按攻击类型聚类的特点，文章提出基于对比语言和图像预训练模型的物理和数字人脸攻击联合检测方法。首先，文章基于混合专家结构提出自适应特征提取模块，通过稀疏激活并结合共享分支实现攻击类型自适应的特征选择；然后，提出一种与攻击无关的可学习文本提示，探索物理和数字攻击的共性伪造线索，实现不同攻击特征簇的有效聚合；最后，引入残差自注意力机制，并设计了细粒度对齐损失，优化共性伪造线索提取过程。在UniAttackData和JFSFDB数据集的联合训练协议上的实验结果表明，相较于其他算法，该方法实现了较低的平均分类错误率。

关键词: 联合攻击检测, 注意力机制, 局部特征, 深度学习, 提示调优

Abstract:

In practical applications, facial recognition systems face the dual threats of physical attacks and digital attacks. Due to the significant heterogeneity between these two types of attacks, different models are often relied upon to address them separately. To conserve computational resources and reduce hardware deployment costs, this paper proposed a joint physical and digital face attack detection method based on the contrastive language and image pre-training model, targeting the characteristic of notable distribution differences and clustering by attack type in the feature space. Firstly, an adaptive feature extraction module was proposed based on the mixture of experts (MoE) structure, achieving attack-type-adaptive feature selection through sparse activation combined with a shared branch; Secondly, an attack-agnostic learnable text prompt was proposed to explore the common forgery clues of physical and digital attacks, enabling effective aggregation of different attack feature clusters; Finally, a residual self-attention mechanism was introduced, and a fine-grained alignment loss was designed to optimize the extraction process of common forgery clues. Experimental results under the joint training protocol on the UniAttackData and JFSFDB datasets show that the proposed method achieves the lower average classification error rate (ACER) compared to other algorithms.

Key words: unified attack detection, attention mechanism, local features, deep learning, prompts tuning

中图分类号:

TP309

梁凤梅, 潘正豪, 刘阿建. 基于共性伪造线索感知的物理和数字人脸攻击联合检测方法[J]. 信息网络安全, 2025, 25(10): 1604-1614.

LIANG Fengmei, PAN Zhenghao, LIU Ajian. A Joint Detection Method for Physical and Digital Face Attacks Based on Common Forgery Clue Awareness[J]. Netinfo Security, 2025, 25(10): 1604-1614.

图/表 10

图1

图2

表1

表2

表3

表4

表 5

表6

图3

图4

参考文献 26

[1]	SUN Zhenan, HE Ran, WANG Liang, et al. Overview of Biometrics Research[J]. Journal of Image and Graphics, 2021, 26(6): 1254-1329. doi: 10.11834/jig.210078 URL
	孙哲南, 赫然, 王亮, 等. 生物特征识别学科发展报告[J]. 中国图象图形学报, 2021, 26(6):1254-1329.
[2]	ZHANG Zhiwei, YAN Junjie, LIU Sifei, et al. A Face Anti-Spoofing Database with Diverse Attacks[C]// IEEE. 5th IAPR International Conference on Biometrics (ICB). New York: IEEE, 2012: 26-31.
[3]	CHINGOVSKA I, ANJOS A, MARCEL S. On the Effectiveness of Local Binary Patterns in Face Anti-Spoofing[C]// IEEE. BIOSIG-Proceedings of the International Conference of Biometrics Special Interest Group (BIOSIG). New York: IEEE, 2012: 1-7.
[4]	LIU Ajian, ZHAO Chenxu, YU Zitong, et al. Contrastive Context-Aware Learning for 3D High-Fidelity Mask Face Presentation Attack Detection[J]. IEEE Transactions on Information Forensics and Security, 2022, 17: 2497-2507. doi: 10.1109/TIFS.2022.3188149 URL
[5]	CIFTCI U A. Fakecatcher: Detection of Synthetic Portrait Videos Using Biological Signals[EB/OL]. (2020-07-15)[2025-03-13]. https://ieeexplore.ieee.org/abstract/document/9141516.
[6]	TOLOSANA R, VERA-RODRIGUEZ R, FIERREZ J, et al. Deepfakes and Beyond: A Survey of Face Manipulation and Fake Detection[J]. Information Fusion, 2020, 64: 131-148. doi: 10.1016/j.inffus.2020.06.014 URL
[7]	YU Zitong, CAI Rizhao, CUI Yawen, et al. Visual Prompt Flexible-Modal Face Anti-Spoofing[J]. IEEE Transactions on Dependable and Secure Computing, 2024, 22: 2597-2606. doi: 10.1109/TDSC.2024.3520534 URL
[8]	WANG Chundong, LI Quan, FU Haoran, et al. Face Anti-Spoofing Method with Adversarial Robustness[EB/OL]. (2024-06-06) [2025-03-13]. https://www.jsjkx.com/CN/10.11896/jsjkx.230400022.
	王春东, 李泉, 付浩然, 等. 具有对抗鲁棒性的人脸活体检测方法[EB/OL]. (2024-06-06)[2025-03-13]. https://www.jsjkk.com/CN/10.11896/jsjkk.230400022.
[9]	ZHOU Yansen, XU Chuankai, CUI Jianquan. Face Recognition in Vivo Based on Temporal Optical Flow and Micro-Expression[J]. Computer Applications and Software, 2024, 41(12): 188-192.
	周延森, 徐传凯, 崔见泉. 基于时序光流与微表情的人脸活体识别[J]. 计算机应用与软件, 2024, 41(12):188-192.
[10]	AGARWAL A. MagNet: Detecting Digital Presentation Attacks on Face Recognition[EB/OL]. (2021-12-08)[2025-03-13]. https://www.frontiersin.org/journals/artificial-intelligence/articles/10.3389/frai.2021.643424/full.
[11]	ZHU Xintong, TANG Yunqi, GENG Pengzhi. Detection Algorithm of Tamper and Deepfake Image Based on Feature Fusion[J]. Netinfo Security, 2021, 21(8): 70-81.
	朱新同, 唐云祁, 耿鹏志. 基于特征融合的篡改与深度伪造图像检测算法[J]. 信息网络安全, 2021, 21(8):70-81.
[12]	XU Kaiwen, ZHOU Yichao, GU Wenquan, et al. A Multi-Scale Feature Fusion Deepfake Detection Algorithm Based on Reconstruction Learning[J]. Netinfo Security, 2024, 24(8): 1173-1183.
	许楷文, 周翊超, 谷文权, 等. 基于多尺度特征融合重建学习的深度伪造人脸检测算法[J]. 信息网络安全, 2024, 24(8):1173-1183.
[13]	DEB D, LIU Xiaoming, JAIN A K. Unified Detection of Digital and Physical Face Attacks[C]// IEEE. 17th International Conference on Automatic Face and Gesture Recognition (FG). New York: IEEE, 2023: 1-8.
[14]	YU Zitong, CAI Rizhao, LI Zhi, et al. Benchmarking Joint Face Spoofing and Forgery Detection with Visual and Physiological Cues[J]. IEEE Transactions on Dependable and Secure Computing, 2024, 21(5): 4327-4342. doi: 10.1109/TDSC.2024.3352049 URL
[15]	FANG Hao, LIU Ajian, YUAN Haocheng, et al. Unified Physical-Digital Face Attack Detection[C]// IJCAI. The Thirty-Third International Joint Conference on Artificial Intelligence. San Francisco: Morgan Kaufmann, 2024: 749-757.
[16]	ZOU Hang, DU Chenxi, ZHANG Hui, et al. La-SoftMoE CLIP for Unified Physical-Digital Face Attack Detection[C]// IEEE. 2024 IEEE International Joint Conference on Biometrics (IJCB). New York: IEEE, 2024: 1-11.
[17]	CHEN Shunxin, LIU Ajian, ZHENG Junze, et al. Mixture-of-Attack-Experts with Class Regularization for Unified Physical-Digital Face Attack Detection[C]// AAAI. Proceedings of the AAAI Conference on Artificial Intelligence. Menlo Park: AAAI, 2025: 2195-2203.
[18]	RADFORD A, KIM J W, HALLACY C, et al. Learning Transferable Visual Models from Natural Language Supervision[C]// IMLS. The 38th International Conference on Machine Learning. Massachusetts: PMLR, 2021: 8748-8763.
[19]	DAI Damai, DENG Chengqi, ZHAO Chenggang, et al. DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models[C]// ACL. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2024: 1280-1297.
[20]	DIKO A. ReViT: Enhancing Vision Transformers Feature Diversity with Attention Residual Connections[EB/OL]. (2024-12-01)[2025-03-13]. https://www.sciencedirect.com/science/article/abs/pii/S0031320324006046.
[21]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep Residual Learning for Image Recognition[C]// IEEE. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2016: 770-778.
[22]	DOSOVITSKIY A. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale[EB/OL]. (2020-11-22)[2025-03-13]. https://arxiv.org/abs/2010.11929.
[23]	DANG Hao, LIU Feng, STEHOUWWER J, et al. On the Detection of Digital Face Manipulation[C]// IEEE. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern recognition. New York: IEEE, 2020: 5781-5790.
[24]	YU Zitong, ZHAO Chenxu, WANG Zezheng, et al. Searching Central Difference Convolutional Networks for Face Anti-Spoofing[C]// IEEE. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2020: 5295-5305.
[25]	LIU Yaojie, JOURABLOO A, LIU Xiaoming. Learning Deep Models for Face Anti-Spoofing: Binary or Auxiliary Supervision[C]// IEEE. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2018: 389-398.
[26]	CAI T T, MA Rong. Theoretical Foundations of T-SNE for Visualizing High-Dimensional Clustered Data[J]. Journal of Machine Learning Research, 2022, 23(301): 1-54.

类别	训练样本数/张	验证样本数/张	测试样本数/张
对象数量	600	300	900
真实样本	3000	1500	4500
物理攻击	1800	900	2700
对抗攻击	1800	1800	7106
数字攻击	1800	1800	7200
总数	8400	6000	21506

方法	ACER	ACC	AUC	EER
ResNet50	1.35%	98.83%	99.79%	1.18%
ViT-B/16	5.92%	92.29%	97.00%	9.14%
Auxiliary	1.13%	98.68%	99.82%	1.23%
CDCN	1.40%	98.57%	99.52%	1.42%
FFD	2.01%	97.97%	99.57%	2.01%
UniAttackDetect	0.52%	99.45%	99.95%	0.53%
La-SoftMoE	0.32%	99.54%	99.72%	0.56%
MoAE-CR	0.37%	99.47%	99.97%	0.49%
本文方法	0.30%	99.56%	99.82%	0.61%

方法	ACER	ACC	AUC	EER
ResNet50	7.70%	90.43%	98.04%	6.71%
ViT-B/16	8.75%	90.11%	98.16%	7.54%
Auxiliary	11.16%	87.40%	97.39%	9.16%
CDCN	12.31%	86.18%	95.93%	10.29%
FFD	9.86%	89.41%	95.48%	9.98%
La-SoftMoE	4.21%	95.85%	99.11%	4.19%
MoAE-CR	4.40%	95.33%	98.97%	4.66%
本文方法	4.17%	95.89%	99.23%	4.51%

方法	ACER	ACC	AUC	EER
CLIP	0.91%	98.87%	99.76%	0.96%
CLIP+AFE	0.54%	99.33%	99.62%	0.76%
CLIP+AFE+ALTP	0.38%	99.42%	99.73%	0.69%
CLIP+AFE+ALTP+FSE	0.34%	99.45%	99.77%	0.66%
CLIP+AFE+ALTP+RSA+FSE（本文方法）	0.30%	99.56%	99.82%	0.61%

专家网络数量/个	ACER	ACC	AUC	EER
3	0.39%	99.54%	99.76%	0.64%
4	0.30%	99.56%	99.82%	0.61%
5	0.36%	99.52%	99.68%	0.69%
6	0.42%	99.46%	99.77%	0.66%