基于异构数据的联邦学习自适应差分隐私方法研究

doi:10.3969/j.issn.1671-1122.2025.01.006

摘要/Abstract

摘要：

在联邦学习中，由于需要大量的参数交换，可能会引发来自不可信参与设备的安全威胁。为了保护训练数据和模型参数，必须采用有效的隐私保护措施。鉴于异构数据的不均衡特性，文章提出一种自适应性差分隐私方法来保护基于异构数据的联邦学习的安全性。首先为不同的客户端设置不同的初始隐私预算，对局部模型的梯度参数添加高斯噪声；其次在训练过程中根据每一轮迭代的损失函数值，动态调整各个客户端的隐私预算，加快收敛速度；接着设定一个可信的中央节点，对不同客户端的局部模型的每一层参数进行随机交换，然后将混淆过后的局部模型参数上传到中央服务器进行聚合；最后中央服务器聚合可信中央节点上传的混淆参数，根据预先设定的全局隐私预算阈值，对全局模型添加合适的噪声，进行隐私修正，实现服务器层面的隐私保护。实验结果表明，在相同的异构数据条件下，相对于普通的差分隐私方法，该方法具有更快的收敛速度以及更好的模型性能。

关键词: 联邦学习, 异构数据, 差分隐私, 高斯噪声

Abstract:

In federated learning, the need for a large amount of parameter exchange may lead to security threats from untrusted participating devices. In order to protect training data and model parameters, effective privacy protection measures must be taken. Given the imbalanced nature of heterogeneous data, this paper proposed an adaptive differential privacy method to protect the security of federated learning based on heterogeneous data. Firstly, different initial privacy budgets were set for different clients, and Gaussian noise was added to the gradient parameters of the local model; Secondly, during the training process, the privacy budget of each client was dynamically adjusted based on the loss function value of each iteration to accelerate convergence speed; Then, set a trusted central node to randomly exchange the parameters of each layer of local models from different clients, and then uploaded the confused local model parameters to the central server for aggregation; Finally, the central server aggregated the obfuscation parameters uploaded by trusted central nodes, added appropriate noise to the global model based on a pre-set global privacy budget threshold, and performed privacy correction to achieve server level privacy protection. The experimental results show that under the same heterogeneous data conditions, compared to ordinary differential privacy methods, the adaptive differential privacy method proposed in this paper has faster convergence speed and better model performance.

Key words: federated learning, heterogeneous data, differential privacy, Gaussian noise

中图分类号:

TP309

徐茹枝, 仝雨蒙, 戴理朋. 基于异构数据的联邦学习自适应差分隐私方法研究[J]. 信息网络安全, 2025, 25(1): 63-77.

XU Ruzhi, TONG Yumeng, DAI Lipeng. Research on Federated Learning Adaptive Differential Privacy Method Based on Heterogeneous Data[J]. Netinfo Security, 2025, 25(1): 63-77.

图/表 13

图1

图2

图3

图4

图5

表1

图6

图7

图8

表2

表3

图9

图10

参考文献 18

[1]	XIONG Shiqiang, HE Daojing, WANG Zhendong, et al. A Review of Federated Learning and its Security and Privacy Protection[J]. Computer Engineering, 2024, 50(5): 1-15. doi: 10.19678/j.issn.1000-3428.0067782
	熊世强, 何道敬, 王振东, 等. 联邦学习及其安全与隐私保护研究综述[J]. 计算机工程, 2024, 50(5): 1-15. doi: 10.19678/j.issn.1000-3428.0067782
[2]	ACAR A, AKSU H, ULUAGAC A S, et al. A Survey on Homomorphic Encryption Schemes: Theory and Implementation[J]. ACM Computing Surveys (Csur), 2018, 51(4): 1-35.
[3]	LI Xiang. On the Convergence of Fedavg on Non-IID Data[EB/OL]. (2020-06-25)[2024-09-15]. https://doi.org/10.48550/arXiv.1907.02189.
[4]	ZHAO Yue. Federated Learning with Non-IID Data[EB/OL]. (2022-07-21)[2024-09-15]. https://doi.org/10.48550/arXiv.1806.00582.
[5]	ABADI M, CHU A, GOODFELLOW I, et al. Deep Learning with Differential Privacy[C]// ACM. Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. New York: ACM, 2016: 308-318.
[6]	DU Wenliang, ATALLAH M J. Secure Multi-Party Computation Problems and their Applications: A Review and Open Problems[C]// ACM.Proceedings of the 2001 Workshop on New Security Paradigms. New York: ACM, 2001: 13-22.
[7]	HASHEMI H, WANG Yongqin, ANNAVARAM M. DarKnight: An Accelerated Framework for Privacy and Integrity Preserving Deep Learning Using Trusted Hardware[C]// ACM. MICRO-54: 54th Annual IEEE/ACM International Symposium on Microarchitecture. New York: ACM, 2021: 212-224.
[8]	MOHAMMADI N, BAI Jianan, FAN Qiang, et al. Differential Privacy Meets Federated Learning under Communication Constraints[J]. IEEE Internet of Things Journal, 2021, 9(22): 22204-22219.
[9]	GONG Xuan, SONG Liangchen, VEDULA R, et al. Federated Learning with Privacy-Preserving Ensemble Attention Distillation[J]. IEEE Transactions on Medical Imaging, 2022, 42(7): 2057-2067.
[10]	GAO Dashan, LIU Yang, HUANG Anbu, et al. Privacy-Preserving Heterogeneous Federated Transfer Learning[C]// IEEE. 2019 IEEE International Conference on Big Data (Big Data). New York: IEEE, 2019: 2552-2559.
[11]	NOBLE M, BELLET A, DIEULEVEUT A. Differentially Private Federated Learning on Heterogeneous Data[C]// PMLR. International Conference on Artificial Intelligence and Statistics. New York: PMLR, 2022: 10110-10145.
[12]	YANG Li, ZHU Lingbo, YU Yueming, et al. Review of Federal Learning and Offensive-Defensive Confrontation[J]. Netinfo Security, 2023, 23(12): 69-90.
	杨丽, 朱凌波, 于越明, 等. 联邦学习与攻防对抗综述[J]. 信息网络安全, 2023, 23(12): 69-90.
[13]	MAMMEN P M. Federated Learning: Opportunities and Challenges[EB/OL]. (2021-01-14)[2024-09-15]. https://doi.org/10.48550/arXiv.2101.05428.
[14]	YANG Liuyan, HE Juanjuan, FU Yue, et al. Federated Learning for Medical Imaging Segmentation via Dynamic Aggregation on Non-IID Data Silos[J]. Electronics, 2023, 12(7): 1687-1707.
[15]	LI Qinbin, DIAO Yiqun, CHEN Quan, et al. Federated Learning on Non-IId Data Silos: An Experimental Study[C]// IEEE. 2022 IEEE 38th International Conference on Data Engineering (ICDE). New York: IEEE, 2022: 965-978.
[16]	DWORK C. Differential Privacy[C]// Springer. International Colloquium on Automata, Languages, and Programming. Heidelberg: Springer, 2006: 1-12.
[17]	XU Ruzhi, DAI Lipeng, XIA Diya, et al. Research on Centralized Differential Privacy Algorithm for Federated Learning[J]. Netinfo Security, 2024, 24(1): 69-79.
	徐茹枝, 戴理朋, 夏迪娅, 等. 基于联邦学习的中心化差分隐私保护算法研究[J]. 信息网络安全, 2024, 24(1): 69-79.
[18]	GIRGIS A M, DATA D, DIGGAVI S, et al. Shuffled Model of Federated Learning: Privacy, Accuracy and Communication Trade-Offs[J]. IEEE Journal on Selected Areas in Information Theory, 2021, 2(1): 464-478.

名称	配置信息
操作系统	Windows 10
CPU	Intel(R) Core(TM) i7-7500U
GPU	NVIDIA RTX 4070(24 GB)
框架	PyTorch 1.10.0+cuda11.1
内存	16 GB
开发语言	Python 3.8

客户端	设定1	设定2	设定3	设定4
MNIST	1	2	4	8
SVHN	1	2	4	8
USPS	2	4	8	16
SynthDigits	2	4	8	16
MNIST-M	3	6	12	24
总隐私预算	9	18	36	72
隐私阈值	9	18	36	72

客户端	设定1	设定2	设定3	设定4
MNIST	79.23%	87.51%	95.47%	97.36%
SVHN	38.63%	55.31%	70.27%	78.98%
USPS	75.81%	86.32%	95.23%	96.64%
SynthDigits	65.72%	80.28%	90.29%	93.61%
MNIST-M	43.78%	59.04%	71.78%	76.54%