Abstract:

In practical applications, facial recognition systems face the dual threats of physical attacks and digital attacks. Due to the significant heterogeneity between these two types of attacks, different models are often relied upon to address them separately. To conserve computational resources and reduce hardware deployment costs, this paper proposed a joint physical and digital face attack detection method based on the contrastive language and image pre-training model, targeting the characteristic of notable distribution differences and clustering by attack type in the feature space. Firstly, an adaptive feature extraction module was proposed based on the mixture of experts (MoE) structure, achieving attack-type-adaptive feature selection through sparse activation combined with a shared branch; Secondly, an attack-agnostic learnable text prompt was proposed to explore the common forgery clues of physical and digital attacks, enabling effective aggregation of different attack feature clusters; Finally, a residual self-attention mechanism was introduced, and a fine-grained alignment loss was designed to optimize the extraction process of common forgery clues. Experimental results under the joint training protocol on the UniAttackData and JFSFDB datasets show that the proposed method achieves the lower average classification error rate (ACER) compared to other algorithms.

Key words: unified attack detection, attention mechanism, local features, deep learning, prompts tuning