Privacy Protection Scheme of Feedforward Neural Network Based on Homomorphic Encryption

doi:10.3969/j.issn.1671-1122.2024.09.006

Abstract

Abstract:

The current privacy-preserving machine learning (PPML) method has made certain progress in ensuring data privacy, but it still faces challenges in terms of computing efficiency and server resource utilization. In order to make full use of server resources and for feedforward neural network, this paper proposed a privacy protection scheme for homomorphic encryption feedforward neural network based on a master-slave server architecture. This scheme used secret sharing technology to distribute data and model parameters to two non-colluding servers, and used homomorphic encryption technology to encrypt interactive information between servers. In terms of computational efficiency, the running time of the scheme was reduced by avoiding running ciphertext management and plaintext matrix multiplication. In terms of security, adding noise to secret sharing by introducing random noise prevents the server from obtaining original data information. The experimental results show that the proposed scheme has improved both in computational complexity and communication overhead.

Key words: homomorphic encryption, neural network, privacy protection, secret sharing

CLC Number:

TP309

LIN Zhanhang, XIANG Guangli, LI Zhenpeng, XU Ziyi. Privacy Protection Scheme of Feedforward Neural Network Based on Homomorphic Encryption[J]. Netinfo Security, 2024, 24(9): 1375-1385.

Figures/Tables 9

References 20

[1]	HAMDAN M A, MAKLOUF A M, MNIF H. Review of Authentication with Privacy-Preserving Schemes for 5G-Enabled Vehicular Networks[C]// IEEE. 2022 15th International Conference on Security of Information and Networks. New York: IEEE, 2022, 15(4): 1-6.
[2]	RANGARAJU S, NESS S, DHARMALINGAM R. Incorporating AI-Driven Strategies in DevSecOps for Robust Cloud Security[J]. International Journal of Innovative Science and Research Technology, 2023, 8(23): 2359-2365.
[3]	TANUWIDJAJA H C, CHOI R, BAEK S, et al. Privacy-Preserving Deep Learning on Machine Learning as a Service-a Comprehensive Survey[J]. IEEE Access, 2020, 8(2): 167425-167447.
[4]	YANG Wencheng, WANG Song, CUI Hui, et al. A Review of Homomorphic Encryption for Privacy-Preserving Biometrics[EB/OL].(2023-03-29)[2024-04-30]. https://doi.org/10.3390/s23073566.
[5]	WANG Jianhua, LI Lin, ZHAO Zhendong, et al. A Neural Network Prediction Scheme Based on Total Homomorphic Encryption Algorithm[J]. Artificial Intelligence, 2022, 3(4): 97-108.
	王建华, 黎琳, 赵镇东, 等. 一种基于全同态加密算法的神经网络预测方案[J]. 人工智能, 2022, 3(4):97-108.
[6]	GILAD-BACHRACH R, DOWLIN N, LAINE K, et al. Cryptonets: Applying Neural Networks to Encrypted Data with High Throughput and Accuracy[C]// PMLR. International Conference on Machine Learning. New York: PMLR, 2016, 48(2): 201-210.
[7]	HESAMIFARD E, TAKABI H, GHASEMI M. Cryptodl: Towards Deep Learning over Encrypted Data[C]// ACSAC. Annual Computer Security Applications Conference. New York: ACM, 2016, 6(3): 11-21.
[8]	MOHASSEL P, ZHANG Yupeng. SecureML: A System for Scalable Privacy-Preserving Machine Learning[C]// IEEE. 2017 IEEE Symposium on Security and Privacy. New York: IEEE, 2017, 36(2): 19-38.
[9]	LIU Jian, JUUTI M, LU Yao, et al. Oblivious Neural Network Predictions via Minionn Transformations[C]// ACM. Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. New York: ACM, 2017, 14(6): 619-631.
[10]	MOHASSEL P, RINDAL P. ABY3: A Mixed Protocol Framework for Machine Learning[C]// ACM. Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. New York: ACM, 2018, 21(3): 35-52.
[11]	JUVEKAR C, VAIKUNTANATHAN V, CHANDRAKASAN A. Gazelle: A Low Latency Framework for Secure Neural Network Inference[C]// USENIX. 27th USENIX Security Symposium. Berkeley: USENIX, 2018, 23(2): 1651-1669.
[12]	WAGH S, GUPTA D, CHANDRAN N. SecureNN: 3-Party Secure Computation for Neural Network Training[J]. Proceedings on Privacy Enhancing Technologies, 2019, 2019(3): 26-49.
[13]	LI Jinguo, YAN Yan, ZHANG Kai, et al. FPCNN: A Fast Privacy-Preserving Outsourced Convolutional Neural Network with Low-Bandwidth[EB/OL]. (2023-11-01)[2024-04-30]. https://api.semanticscholar.org/CorpusID:265169136.
[14]	REN Yanli, YU Lingzan, HE Gang, et al. A Scheme of Privacy-Preserving Convolutional Neural Network Prediction[J]. Chinese Journal of Computers, 2023, 46(8): 1606-1619.
	任艳丽, 余凌赞, 何港, 等. 一种隐私保护的卷积神经网络预测方案[J]. 计算机学报, 2023, 46(8):1606-1619.
[15]	NARESH V S, THAMARAI M. Privacy-Preserving Data Mining and Machine Learning in Healthcare: Applications, Challenges, and Solutions[J]. (2023-01-24)[2024-04-30]. https://wires.onlinelibrary.wiley.com/doi/abs/10.1002/widm.1490.
[16]	ZHAO Xia, WANG Limin, ZHANG Yufei, et al. A Review of Convolutional Neural Networks in Computer Vision[J]. Artificial Intelligence Review, 2024, 57(4): 1-43.
[17]	DONG Aoxiang, STARR A, ZHAO Yifan. Neural Network-Based Parametric System Identification: A Review[J]. International Journal of Systems Science, 2023, 54(13): 2676-2688.
[18]	CHEON J H, KIM A, KIM M, et al. Homomorphic Encryption for Arithmetic of Approximate Numbers[C]// Springer. Advances in Cryptology-ASIACRYPT 2017: 23rd International Conference on the Theory and Applications of Cryptology and Information Security. Heidelberg: Springer, 2017, 1(23): 409-437.
[19]	CHATTOPADHYAY A K, SAHA S, NAG A, et al. Secret Sharing: A Comprehensive Survey, Taxonomy and Applications[EB/OL]. (2024-02-01)[2024-04-30]. https://doi.org/10.1016/j.cosrev.2023.100608.
[20]	XIA Zhihua, GU Qi, XIONG Lizhi, et al. Privacy-Preserving Image Retrieval Based on Additive Secret Sharing[J]. International Journal of Autonomous and Adaptive Communications Systems, 2024, 17(2): 99-126.

方法名称	方案	数据集	准确率	推理时间/s
CryptoNets	YASHE	MNIST	98.95%	297.500
CryptoDL	BGV	MNIST, CIFAR-10	99.52%	320.000
SecureML	YASHE	MNIST, Gisette	93.40%	4.880
MiniONN	YASHE	MNIST	98.95%	9.320
SecureNN	多方安全计算	MNIST	99.15%	0.230
FPCNN	CKKS	MNIST, CIFAR-10	98.50%	5.095

网络层次	描述
输入层	接收分辨率为28×28像素的图像输入，输出维度为128
激活函数层1	使用非多项式近似的激活函数
隐藏层1	输入维度为128，输出维度为64
激活函数层2	使用非多项式近似的激活函数
隐藏层2	输入维度为64，输出维度为10

网络层协议	计算复杂度	通信开销
输入层	$O(n\text{ }\!\!\times\!\!\text{ }{{N}_{c}})$	$3{{N}_{c}}+P$
激活函数层	$O(n\times {{N}_{A}})$	$3{{N}_{A}}$
隐藏层	$O(n\times m)$	$3{{N}_{c}}$
Softmax层	$O(n({{N}_{c}}+{{N}_{A}}))$	$3{{N}_{c}}+2{{N}_{A}}+P$
合计	$O(n({{N}_{c}}+{{N}_{A}}))$	$9{{N}_{c}}+5{{N}_{A}}+2P$

同态加密操作算法	操作数/次数
同态加密操作算法	乘法	加密	解密	传输
SecureML	152	78	62	143
MiniONN	25	25	25	50
本文方案	11	10	5	15

方案	运行时间/s	通信量大小/MB	准确率
SecureML-Minist	5.12	645.19	91.16%
MiniONN-Minist	1.05	15.80	94.63%
本文方案-Minist	0.42	1.89	96.62%
SecureML-Cifar-10	26.07	3450.75	79.17%
本文方案-Cifar-10	0.62	2.27	83.84%