A New Parameter Masking Federated Learning Privacy Preserving Scheme

doi:10.3969/j.issn.1671-1122.2021.08.004

Abstract

Abstract:

With the successive promulgation of data privacy protection laws and regulations, the problem of privacy data exposure in the traditional centralized learning model has become an important factor restricting the development of artificial intelligence. The proposal of federated learning solves this problem, however, existing federated learning has problems such as model parameters leaking sensitive information and relying on trusted third-party servers. This paper proposed a new parameter masking federated learning privacy preserving scheme, which can resist server attacks, user attacks, server colluding with less than t users attacks. The scheme included three protocols: key exchange, parameter masking, and disconnection processing. User uploaded the masked model parameters after training the model locally. After the server aggregated model parameters, it can only obtain the masked parameter aggregation results. Experiments show that for 16-byte input values, our protocol offer 1.44× communication expansion for 2⁷ user and 2²⁰- dimensional vector over sending data in the clear, and compared with existing scheme, it has lower communication cost.

Key words: federated learning, privacy preserving, parameter masking, user disconnection

CLC Number:

TP309

LU Honglin, WANG Liming, YANG Jing. A New Parameter Masking Federated Learning Privacy Preserving Scheme[J]. Netinfo Security, 2021, 21(8): 26-34.

Figures/Tables 4

References 21

[1]	CUSTERS B, SEARS A, DECHESNE F, et al. Information Technology and Law Series[M]. Netherlands: T. M. C. Asser Press, 2019.
[2]	The National People's Congress and its Standing Committee. The Cybersecurity Law of the People’s Republic of China[EB/OL]. http://www.npc.gov.cn/wxzl/gongbao/2017-02/20/content_2007531.htm, 2021-02-12 .
[3]	YANG Qiang. Federated Learning: the Last Mile of Artificial Intelligence[J]. Journal of Intelligent Systems, 2020, 15(81):189-192.
	杨强. 联邦学习:人工智能的最后一公里[J]. 智能系统学报, 2020, 15(81):189-192.
[4]	SHOKRI R, STRONATI M, SONG Congzheng, et al. Membership Inference Attacks Against Machine Learning Models[C]//IEEE. IEEE Symposium on Security and Privacy, May 22-26, 2017, San Jose, CA, USA. New York: IEEE, 2017: 3-18.
[5]	MELIS L, SONG Congzheng, CRISTOFARO E D, et al. Exploiting Unintended Feature Leakage in Collaborative Learning[C]// IEEE. 2019 IEEE Symposium on Security and Privacy (SP), May 19-23, 2019, San Francisco, CA. New York: IEEE, 2019: 691-706.
[6]	HITAJ B, ATENIESE G, PEREZ-CRUZ F. Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning[C]//ACM. Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, October 30-November 3, 2017, Dallas Texas, USA. New York: ACM, 2017: 603-618.
[7]	WANG Zhibo, SONG Mengkai, ZHANG Zhifei, et al. Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning[C]//IEEE. IEEE INFOCOM 2019-IEEE Conference on Computer Communications, April 29-May 2, 2019, Paris, France. New York: IEEE, 2019: 2512-2520.
[8]	NASR M, SHOKRI R, HOUMANSADR A. Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning[C]//IEEE. IEEE Symposium on Security and Privacy (SP), May 19-23, 2019, San Francisco, CA, USA. New York: IEEE, 2019: 739-753.
[9]	LIM W, LUONG N, HOANG D, et al. Federated Learning in Mobile Edge Networks: A Comprehensive Survey[J]. IEEE Communications Surveys & Tutorials, 2020, 22(3):2031-2063.
[10]	GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative Adversarial Networks[J]. Advances in Neural Information Processing Systems, 2014, 14(3):2672-2680.
[11]	PHONG L, AONO Y, HAYASHI T, et al. Privacy-preserving Deep Learning via Additively Homomorphic Encryption[J]. IEEE Transactions on Information Forensics and Security, 2018, 13(5):1333-1345. doi: 10.1109/TIFS.2017.2787987 URL
[12]	ZHANG Qiao, WANG Cong, WU Hongyi, et al. GELU-net: A Globally Encrypted, Locally Unencrypted Deep Neural Network for Privacy-preserved Learning[C]// Morgan Kaufmann. Twenty-seventh International Joint Conference on Artificial Intelligence IJCAI, July 13-19, 2018, Stockholm, Sweden. California: Morgan Kaufmann, 2018: 3933-3939.
[13]	MA X, ZHANG F, CHEN X, et al. Privacy Preserving Multi-party Computation Delegation for Deep Learning in Cloud Computing[J]. Information Sciences, 2018, 45(9):103-116.
[14]	HAO Meng, LI Hongwei, XU Guowen, et al. Towards Efficient and Privacy-preserving Federated Deep Learning[C]// IEEE. IEEE International Conference on Communications (ICC), May 20-24, 2019, Shanghai, China. New York: IEEE, 2019: 1-6.
[15]	BONAWITZ K, IVANOV V, KREUTER B, et al. Practical Secure Aggregation for Privacy-preserving Machine Learning[C]//ACM. Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, October 30-November 3, 2017, Dallas Texas, USA. New York: ACM, 2017: 1175-1191.
[16]	DUAN Jia, ZHOU Jiantao, LI Yuanman. Privacy-preserving Distributed Deep Learning Based on Secret Sharing[J]. Information Sciences, 2020, 52(7):108-127.
[17]	PYRGELIS A, TRONCOSO C, CRISTOFARO E. Knock Knock. Knock Knock, Who’s There? Membership Inference on Aggregate Location Data[EB/OL]. https://www.researchgate.net/publication/323248809_Knock_Knock_Who’s_There_Membership_Inference_on_Aggregate_Location_Data , 2021-02-01.
[18]	KOCHER P. Timing Attacks on Implementations of Diffie-Hellman, RSA, DSS, and Other Systems[C]//ACM. Proceedings of the 16th Annual International Cryptology Conference on Advances in Cryptology, August 18-22, 1996, Berlin, Germany. New York: ACM, 1996: 104-113.
[19]	SHAMIR A. How to Share a Secret[J]. Commun. ACM, 1979, 22(11):612-613. doi: 10.1145/359168.359176 URL
[20]	LI Hongwei, LIU Dongxiao, DAI Yuanshun, et al. Engineering Searchable Encryption of Mobile Cloud Networks: When QoE Meets QoP[J]. IEEE Wireless Communications, 2015, 22(4):74-80.
[21]	BELLARE M, YEE B. Forward-security in Private-key Cryptography[C]//Springer. Proceedings of the 2003 RSA Conference on The Cryptographers’ Track, April 13-17, 2003, Berlin, Germany. Berlin: Springer, 2003: 1-18.

符号	说明
m	用户个数
d	掉线用户个数
P,${{P}_{d}}$、${{P}_{po}}$、${{P}_{j}}$	用户集合,掉线、延迟上传、普通用户
g	密钥生成元
q	密钥交换协议中的素数
$\alpha $	额外的附加掩盖值
w	模型参数
${{\hat{w}}_{j}}$	本文中掩盖的模型参数
${{\dot{w}}_{j}}$	文献[15]中掩盖的模型参数
${{b}_{j}}$	文献[15]中用户生成的随机数
${{S}_{j,i}}$	用户${{P}_{j}}$和${{P}_{i}}$的协商密钥
t	秘密共享门限值
$P{{K}_{j}}S{{K}_{j}}$	用户${{P}_{j}}$的公钥、私钥
R	PRG的输出最大值
${{R}_{U}}$	原始模型参数的输出最大值
z	PRG的输出维度
${{a}_{K}}$	密钥交换公钥比特数
${{a}_{S}}$	秘密共享份额比特数