基于掩码的选择性联邦蒸馏方案

doi:10.3969/j.issn.1671-1122.2025.06.007

摘要/Abstract

摘要：

随着机器学习技术的不断进步，隐私保护问题越来越被重视。联邦学习作为一种分布式机器学习框架，得到了广泛应用，然而，其在实际应用中仍面临隐私泄露和低效率的挑战。为了应对上述挑战，文章提出基于掩码的选择性联邦蒸馏（MSFD）方案，该方案利用联邦蒸馏通过传递知识而非模型参数的特点，有效抵御白盒攻击，同时降低通信开销。通过在共享的软标签中引入AES加密后的掩码机制，有效解决选择性联邦蒸馏明文共享软标签易受黑盒攻击威胁的问题，显著提升对黑盒攻击的抵御能力，从而大幅提高选择性联邦蒸馏方案的安全性。通过在客户端软标签中嵌入动态加密掩码，实现隐私混淆，并结合秘密信道协商与轮次密钥更新机制，显著降低遭受黑盒攻击的风险并保持模型性能，兼顾联邦学习的安全性与通信效率。通过安全性分析和实验结果表明，MSFD在多个数据集上能够显著降低黑盒攻击成功率，同时保持分类准确率，有效提升隐私保护能力。

关键词: 联邦学习, 知识蒸馏, 掩码机制, 联邦蒸馏, 隐私保护

Abstract:

With the continuous advancement of machine learning technology, privacy protection issues are becoming increasingly important. Federated learning, as a distributed machine learning framework, has been widely applied. However, it still faces challenges in terms of privacy leakage and efficiency in practical applications. In order to address the above challenges, the article proposed masking-based selective federated distillation (MSFD), which utilizes the characteristic of knowledge transfer rather than model parameters in federated distillation to effectively resist white box attacks and reduce communication overhead. By introducing AES encrypted masking mechanism into the shared soft tags, the problem of selective federated distillation plaintext shared soft tags being vulnerable to black box attacks was effectively solved, significantly improving the resistance to black box attacks and thus significantly enhancing the security of selective federated distillation schemes. By embedding dynamic encryption masked in client soft tags to achieve privacy obfuscation, and combining secret channel negotiation and round key update mechanisms, the risk of black box attacks was significantly reduced while maintaining model performance, balancing the security and communication efficiency of federated learning. Through security analysis and experimental results, it has been shown that MSFD can significantly reduce the success rate of black box attacks on multiple datasets, while maintaining classification accuracy and effectively improving privacy protection capabilities.

Key words: federated learning, knowledge distillation, mask mechanism, federated distillation, privacy protection

中图分类号:

TP309

朱率率, 刘科乾. 基于掩码的选择性联邦蒸馏方案[J]. 信息网络安全, 2025, 25(6): 920-932.

ZHU Shuaishuai, LIU Keqian. A Masking-Based Selective Federated Distillation Scheme[J]. Netinfo Security, 2025, 25(6): 920-932.

图/表 10

图1

表1

表2

表3

表4

图2

图3

图4

图5

表5

参考文献 20

[1]	MCMAHAN B H, MOORE E, RAMAGE D, et al. Communication-Efficient Learning of Deep Networks from Decentralized Data[C]// JMLR. Artificial intelligence and statistics. New York: JMLR, 2017: 1273-1282.
[2]	RAJKOMAR A, DEAN J, KOHANE I. Machine Learning in Medicine[J]. New England Journal of Medicine, 2019, 380(14): 1347-1358. doi: 10.1056/NEJMra1814259
[3]	SUN Yu, LIU Zheng, CUI Jian, et al. Client-Side Gradient Inversion Attack in Federated Learning Using Secure Aggregation[J]. IEEE Internet of Things Journal, 2024, 11 (17): 28774-28786.
[4]	QI Tao, WU Fangzhao, WU Chuhan, et al. Differentially Private Knowledge Transfer for Federated Learning[EB/OL]. (2023-05-15)[2025-03-20]. https://doi.org/10.1038/s41467-023-38794-x.
[5]	SHAO Jiawei, WU Fangzhao, ZHANG Jun. Selective Knowledge Sharing for Privacy-Preserving Federated Distillation without a Good Teacher[EB/OL]. (2023-12-12)[2025-03-20]. https://doi.org/10.1038/s41467-023-44383-9.
[6]	HU Hongsheng, SALCIC Z, SUN Lichao, et al. Membership Inference Attacks on Machine Learning: A Survey[J]. ACM Computing Surveys, 2022, 54 (11s): 1-37.
[7]	LIU Lumin, ZHANG Jun, SONG Shuhang, et al. Communication-Efficient Federated Distillation with Active Data Sampling[C]// IEEE. ICC2022-IEEE International Conference on Communications. New York: IEEE, 2022: 201-206.
[8]	HAN Yu. A Review of Knowledge Distillation in Deep Neural Networks[J]. Computer Science and Applications, 2020, 10(9): 1625-1630.
[9]	VOIGT P, VON D B A. The EU General Data Protection Regulation (GDPR)[M]. Heidelberg: Springer, 2017.
[10]	PARK W, KIM D, LU Yan, et al. Relational Knowledge Distillation[C]// IEEE. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2019: 3967-3976.
[11]	GOU Jun, YU Bo, MAYBANK S J, et al. Knowledge Distillation: A Survey[J]. International Journal of Computer Vision, 2021, 129: 1789-1819.
[12]	NGUYEN Q, VALIZADEGAN H, HAUSKRECHT M. Learning Classification Models with Soft-Label Information[J]. Journal of the American Medical Informatics Association, 2014, 21(3): 501-508. doi: 10.1136/amiajnl-2013-001964 pmid: 24259520
[13]	YANG Qiang, LIU Yang, CHEN Tianjian, et al. Federated Machine Learning: Concept and Applications[J]. ACM Transactions on Intelligent Systems and Technology, 2019, 10(2): 1-19.
[14]	ZHU Hangyu, XU Jinjin, LIU Shiqing, et al. Federated Learning on Non-Iid Data: A Survey[J]. Neurocomputing, 2021, 465: 371-390.
[15]	OUADRHIRI A E, ABDELHADI A. Differential Privacy for Deep and Federated Learning: A Survey[J]. IEEE Access, 2022, 10: 22359-22380.
[16]	ITAHARA S, NISHIO T, KODA Y, et al. Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-Iid Private Data[J]. IEEE Transactions on Mobile Computing, 2021, 22: 191-205.
[17]	SHOKRI R, STRONATI M, SONG Congzheng, et al. Membership Inference Attacks against Machine Learning Models[C]// IEEE. IEEE Symposium on Security and Privacy (SP). New York: IEEE, 2017: 3-18.
[18]	LUO Zeren, ZHU Chuangwei, FANG Lujie, et al. An Effective and Practical Gradient Inversion Attack[J]. International Journal of Intelligent Systems, 2022, 37(11): 9373-9389.
[19]	JIANG Yingrui, ZHAO Xuejian, LI Hao, et al. A Personalized Federated Learning Method Based on Knowledge Distillation and Differential Privacy[EB/OL]. (2024-09-06)[2025-03-20]. https://doi.org/10.3390/electronics13173538.
[20]	WU Xiang, ZHANG Yongting, SHI Minyu, et al. An Adaptive Federated Learning Scheme with Differential Privacy Preserving[J]. Future Generation Computer Systems, 2022, 127: 362-372.

实体/模块	复杂度描述	表达式
客户端AES 掩码生成	每个客户端需与所有其他客户端协商密钥，生成动态掩码。每对客户端加密掩码的生成复杂度为AES-CTR的固定计算量	$O({{K}^{2}}\cdot \lambda )$
客户端本地训练	基于本地数据集训练个性化模型，复杂度取决于数据规模与模型结构	$O(D\cdot E)$
服务器软标签聚合	聚合所有客户端的掩码化软标签，执行线性加权平均	$O(K\cdot C)$
服务器知识蒸馏	基于聚合软标签更新全局学生模型，复杂度与模型参数量相关	$O(M)$

实体/模块	通信量描述	表达式
单客户端上传	每轮上传掩码化软标签（类别概率分布）及动态加密掩码种子	$O(C+\lambda )$
全局通信总量	所有客户端上传数据总和，包含掩码化软标签与加密种子	$O(K\cdot (C+\lambda ))$
密钥协商初始化	初始阶段客户端两两协商共享密钥，需安全信道交互	$O({{K}^{2}})$

实体/模块	MSFD	DP	HE
客户端隐私处理	$O({{K}^{2}}\cdot \lambda )$	$O(C+M)$	$O(C\cdot {{\log }^{2}}N)$
服务器聚合	$O(K\cdot C)$	$O(K\cdot C)$	$O({{K}^{3}})$
额外密码学开销	AES-CTR加密	噪声生成与裁剪	密文模逆运算

实体/模块	MSFD	DP	HE
单客户端上传	$O(C+\lambda )$	$O(C)$	$O(C\cdot \log N)$
全局通信总量	$O(K\cdot (C+\lambda ))$	$O(K\cdot C)$	$O(K\cdot C\cdot \log N)$
密钥协商初始化	$O({{K}^{2}})$	无（仅隐私预算协商）	$O(K\cdot \log N)$

数据集	数据分布	Selective-FD	MSFD	HSFD
MNIST	IID	94.54%	93.78%	88.79%
MNIST	Non-IID	92.15%	91.22%	82.45%
Fashion-MNIST	IID	90.78%	89.72%	78.54%
Fashion-MNIST	Non-IID	87.32%	85.91%	70.23%
CIFAR-10	IID	80.23%	78.55%	65.32%
CIFAR-10	Non-IID	77.45%	76.23%	56.41%