Layered Personalized Federated Learning Guided by Model Feature Orientation

doi:10.3969/j.issn.1671-1122.2025.06.004

Abstract

Abstract:

With the rapid advancement of artificial intelligence and the industrial Internet of Things (IIoT), industrial intelligence accelerates. Federated learning (FL) has emerged as a promising solution due to growing privacy concerns. However, FL in IIoT faces challenges like data heterogeneity, resource constraints, and adversarial threats. Existing personalized FL methods customized models for individual clients but ignored aggregation biases during training. To address this issue, this paper proposed a layered personalized FL framework guided by model feature orientations. The framework introduced an efficient aggregation mechanism that captured client-specific information without extra communication costs. It combined an adaptive model quantization mechanism for aggregation weights and a layered dynamic strategy to tailor global models for each client. Experiments on Fashion-MNIST and CWRU datasets show that our approach outperforms baselines in both performance and efficiency.

Key words: industrial internet of things, privacy concerns, personalized federated learning

CLC Number:

TP309

DENG Dongshang, WANG Weiye, ZHANG Weidong, WU Xuangou. Layered Personalized Federated Learning Guided by Model Feature Orientation[J]. Netinfo Security, 2025, 25(6): 889-897.

Figures/Tables 7

References 24

[1]	LI Tian, HU Shengyuan, BEIRAMI A, et al. Ditto: Fair and Robust Federated Learning Through Personalization[C]// PMLR. International Conference on Machine Learning. New York: PMLR, 2021: 6357-6368.
[2]	FALLAH A, MOKHTARI A, OZDAGLAR A. Personalized Federated Learning with Theoretical Guarantees: A Model-Agnostic Meta-Learning Approach[EB/OL]. (2020-12-06)[2024-11-10]. https://dl.acm.org/doi/pdf/10.5555/3495724.3496024.
[3]	DINH C, TRAN N, NGUYEN T D. Personalized Federated Learning with Moreau Envelopes[C]// NIPS. Advances in Neural Information Processing Systems. New York: NIPS, 2020: 21394-21405.
[4]	JIANG Meirui, LE Anjie, LI Xiaoxiao, et al. Heterogeneous Personalized Federated Learning by Local-Global Updates Mixing via Convergence Rate[EB/OL]. (2024-08-07)[2024-11-10]. https://dblp.org/rec/conf/iclr/JiangLL024.html.
[5]	HUANG Yutao, CHU Lingyang, ZHOU Zirui, et al. Personalized Cross-Silo Federated Learning on Non-IID Data[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(9): 7865-7873.
[6]	CHEN Jingxue, YAN Hang, LIU Zhiyuan, et al. When Federated Learning Meets Privacy-Preserving Computation[J]. ACM Computing Surveys, 2024, 56(12): 1-36.
[7]	HUANG Wenke, YE Mang, SHI Zekun, et al. Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmark[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024, 46(12): 9387-9406.
[8]	FU Yuchuan, TANG Xinlong, LI Changle, et al. A Secure Personalized Federated Learning Algorithm for Autonomous Driving[J]. IEEE Transactions on Intelligent Transportation Systems, 2024, 25(12): 20378-20389.
[9]	ZHENG Tianyue, LI Ang, CHEN Zhe, et al. AutoFed: Heterogeneity-Aware Federated Multimodal Learning for Robust Autonomous Driving[C]// ACM. Proceedings of the 29th Annual International Conference on Mobile Computing and Networking. New York: ACM, 2023: 1-15.
[10]	MA Jialiang, TIAN Chunlin, LI Li, et al. FedMG: A Federated Multi-Global Optimization Framework for Autonomous Driving Control[C]// IEEE. 2024 IEEE/ACM 32nd International Symposium on Quality of Service (IWQoS). New York: IEEE, 2024: 1-10.
[11]	JIANG Guoqian, ZHAO Kai, LIU Xiufeng, et al. A Federated Learning Framework for Cloud-Edge Collaborative Fault Diagnosis of Wind Turbines[J]. IEEE Internet of Things Journal, 2024, 11(13): 23170-23185.
[12]	DENG Dongshang, ZHAO Wei, WU Xuangou, et al. DecFFD: A Personalized Federated Learning Framework for Cross-Location Fault Diagnosis[J]. IEEE Transactions on Industrial Informatics, 2024, 20(5): 7082-7091.
[13]	ZHANG Yudong, WANG Xu, WANG Pengkun, et al. Modeling Spatio-Temporal Mobility across Data Silos via Personalized Federated Learning[J]. IEEE Transactions on Mobile Computing, 2024, 23(12): 15289-15306.
[14]	ZHANG Yu, LU Hua, LIU Ning, et al. Personalized Federated Learning for Cross-City Traffic Prediction[EB/OL]. [2024-11-10]. https://doi.org/10.24963/ijcai.2024/611.
[15]	LI Qinbin, HE Bingsheng, SONG D. Model-Contrastive Federated Learning[C]// IEEE. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE, 2021: 10708-10717.
[16]	DURMUS A E, ZHAO Yue, RAMON M, et al. Federated Learning Based on Dynamic Regularization[EB/OL]. (2021-11-09)[2024-11-10]. https://arxiv.org/abs/2111.04263.
[17]	HU Ming, ZHOU Peiheng, YUE Zhihao, et al. FedCross: Towards Accurate Federated Learning via Multi-Model Cross-Aggregation[C]// IEEE. 2024 IEEE 40th International Conference on Data Engineering (ICDE). New York: IEEE, 2024: 2137-2150.
[18]	QI Peihan, ZHOU Xiaoyu, DING Yuanlei, et al. FedBKD: Heterogenous Federated Learning via Bidirectional Knowledge Distillation for Modulation Classification in IoT-Edge System[J]. IEEE Journal of Selected Topics in Signal Processing, 2023, 17(1): 189-204.
[19]	WANG Pengju, LIU Bochao, GUO Weijia, et al. Towards Personalized Federated Learning via Comprehensive Knowledge Distillation[C]// IEEE. 2024 IEEE International Conference on Systems, Man, and Cybernetics (SMC). New York: IEEE, 2024: 3807-3812.
[20]	LUO Jun, WU Shandong. Adapt to Adaptation: Learning Personalization for Cross-Silo Federated Learning[EB/OL]. [2024-11-10]. https://doi.org/10.24963/ijcai.2022/301.
[21]	MCMAHAN H B, MOORE E, RAMAGE D, et al. Communication-Efficient Learning of Deep Networks from Decentralized Data[EB/OL]. (2023-01-26)[2024-11-10]. https://arxiv.org/abs/1602.05629v4.
[22]	DENG Dongshang, WU Xuangou, ZHANG Tao, et al. FedASA: A Personalized Federated Learning with Adaptive Model Aggregation for Heterogeneous Mobile Edge Computing[J]. IEEE Transactions on Mobile Computing, 2024, 23(12): 14787-14802.
[23]	LI Tian, SAHU A K, ZAHEER M, et al. Federated Optimization in Heterogeneous Networks[J]. Proceedings of Machine Learning and Systems, 2020, 2: 429-450.
[24]	ARIVAZHAGAN M G, AGGARWAL V, SINGH A K, et al. Federated Learning with Personalization Layers[EB/OL]. (2019-12-02)[2024-11-10]. https://doi.org/10.48550/arXiv.1912.00818.

方法	Fashion-MNIST		CWRU
方法	$s=3$	$s=5$	两个位置	3个位置
FedAvg	94.32%	88.10%	83.38%	70.22%
FedProx	94.16%	87.86%	81.13%	68.92%
Ditto	94.09%	87.84%	91.00%	84.17%
FedPer	95.47%	91.81%	93.00%	91.42%
本文方法	95.74%	93.82%	95.56%	94.78%