Privacy-preserving Strategies for Federated Learning Based on Data Attribute Modification

doi:10.3969/j.issn.1671-1122.2022.01.007

Abstract

Abstract:

Most defense methods suffer from weak federated learning utility, low computational efficiency, and defense against a single type of attack. To solve the above problems, this paper proposed an attribute modification framework based on variational auto-encoders to achieve the purpose of protecting federated learning by pre-processing the data at the client. First, to improve the computational efficiency of the algorithm and utilize the computational and storage resources of the server, this paper proposed a transfer learning based variational auto-encoders training scheme to reduce the client training epochs. Secondly, to balance practicality and privacy and to utilize the latent variables with continuous properties of the variational auto-encoders, this paper designed an attribute modification scheme based on attribute distribution constraint rules to achieve the reconstruction of client training data. Detailed experimental results show that the attribute modification scheme can successfully separate and control the attribute vectors of an image, protecting client data privacy by changing the original image to a reconstructed image with corresponding attributes. The usability of the scheme is demonstrated by the fact that the images with three modified attributes can be used to train the federated learning classification task with accuracy of 94.44%. And the scheme successfully defends against unintended feature leakage and backdoor attacks based on data poisoning.

Key words: federated learning, privacy protection, VAE, transfer learning

CLC Number:

TP309

XU Shuo, ZHANG Rui, XIA Hui. Privacy-preserving Strategies for Federated Learning Based on Data Attribute Modification[J]. Netinfo Security, 2022, 22(1): 55-63.

Figures/Tables 10

References 17

[1]	DENG Jia, DONG Wei, SOCHER R, et al. Imagenet: A Large-scale Hierarchical Image Database[C]//IEEE. IEEE-computer-society Conference on Computer Vision and Pattern Recognition Workshops, June 20-25, 2009, Miami, FL, USA. New York: IEEE, 2009: 248-255.
[2]	MCMAHAN B, MOORE E, RAMAGE D, et al. Communication-efficient Learning of Deep Networks from Decentralized Data[C]//PMLR. 20th International Conference on Artificial Intelligence and Statistics, April 20-22, 2017, Fort Lauderdale, FL, USA. Brookline: Microtome Publishing, 2017: 1273-1282.
[3]	MELIS L, SONG Congzheng, CRISTOFARO E D, et al. Exploiting Unintended Feature Leakage in Collaborative Learning[C]//IEEE. 40th IEEE Symposium on Security and Privacy (SP), May 19-23, 2019, San Francisco, CA, USA. New York: IEEE, 2019: 691-706.
[4]	WANG Hongyi, SREENIVASAN K, RAJPUT S, et al. Attack of the Tails: Yes, You Really Can Backdoor Federated Learning[J]. NIPS, 2020, 33(5):16070-16084.
[5]	RIVEST R L, ADLEMAN L, DERTOUZOS M L. On Data Banks and Privacy Homomorphisms[J]. Foundations of Secure Computation, 1978, 4(11):169-180.
[6]	LIU Yang, KANG Yan, XING Chaoping, et al. A Secure Federated Transfer Learning Framework[J]. IEEE Intelligent Systems, 2020, 35(4):70-82. doi: 10.1109/MIS.9670 URL
[7]	DWORK C, MCSHERRY F, NISSIM K, et al. Calibrating Noise to Sensitivity in Private Data Analysis[C]//Springer. 3rd Theory of Cryptography Conference, March 4-7, 2006, New York, NY, USA. Berlin: Springer-Verlag, 2006: 265-284.
[8]	JAYARAMAN B, EVANS D. When Relaxations Go Bad: “Differentially- private”Machine Learning[EB/OL]. https://arxiv.org/abs/1902.08874v2, 2019-08-12.
[9]	BLANCHARD P, EL MHAMDI E M, GUERRAOUI R, et al. Machine Learning with Adversaries: Byzantine Tolerant Gradient Descent[C]//NIPS. 31st Annual Conference on Neural Information Processing Systems, December 4-9, 2017, Long Beach, CA, USA. California: NIPS, 2017: 118-128.
[10]	FANG Minghong, CAO Xiaoyu, JIA Jinyuan, et al. Local Model Poisoning Attacks to Byzantine-robust Federated Learning[C]//USENIX. 29th USENIX Security Symposium, August 12-14, 2020, Boston, MA, USA. Berkeley: USENIX ASSOC, 2020: 1605-1622.
[11]	KAIROUZ P, MCMAHAN H B, AVENT B, et al. Advances and Open Problems in Federated Learning[J]. Foundations and Trends in Machine Learning, 2021, 14(1-2):1-210. doi: 10.1561/2200000083 URL
[12]	HE Chaoyang, LI Songze, SO J, et al. Fedml: A Research Library and Benchmark for Federated Machine Learning[EB/OL]. https://arxiv.org/abs/2007.13518, 2020-11-08.
[13]	KINGMA D P, WELLING M. Auto-encoding Variational Bayes[EB/OL]. https://arxiv.org/abs/1312.6114, 2014-05-01.
[14]	XIE Chulin, HUANG Keli, CHEN Pinyu, et al. Dba: Distributed Backdoor Attacks against Federated Learning[C]// ICLR. 7th International Conference on Learning Representations, May 6-9, 2019, New Orleans, LA, USA. New York: ICLR, 2019: 142-151.
[15]	LIU Ziwei, LUO Ping, WANG Xiaogang, et al. Large-scale Celebfaces Attributes (Celeba) Dataset[EB/OL]. http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html, 2021-09-10.
[16]	ROTHE R, TIMOFTE R, VAN GOOL L. Deep Expectation of Real and Apparent Age from a Single Image without Facial Landmarks[J]. International Journal of Computer Vision, 2018, 126(2):144-157. doi: 10.1007/s11263-016-0940-3 URL
[17]	VAHDAT A, KAUTZ J. Nvae: A Deep Hierarchical Variational Autoencoder[EB/OL]. https://arxiv.org/abs/2007.03898, 2021-01-08.

属性阈值	Black_Hair	Smiling	High_Cheekbones	Straight_Hair
$\gamma $	105655	7261	18221	118155
$\iota $	71533	9577	14259	74663
$\tau $	34122	16838	32480	43492

属性	$\beta $	0	1	2	3	4	5	6	7	8	9
Black_Hair	1.0	65%	63%	66%	66%	65%	64%	66%	65%	67%	61%
	0.5	79%	77%	79%	79%	79%	79%	80%	78%	81%	75%
	0.1	86%	84%	85%	84%	86%	86%	85%	83%	85%	83%
Smiling	1.0	56%	56%	56%	57%	56%	56%	56%	56%	56%	54%
	0.5	74%	74%	75%	76%	75%	74%	74%	73%	73%	71%
	0.1	86%	86%	86%	86%	87%	85%	85%	85%	85%	85%
Straight_Hair	1.0	72%	71%	72%	70%	72%	71%	72%	72%	72%	73%
	0.5	76%	75%	75%	74%	76%	74%	76%	74%	75%	76%
	0.1	75%	74%	74%	73%	74%	73%	75%	73%	74%	74%