一种基于双阈值函数的成员推理攻击方法

doi:10.3969/j.issn.1671-1122.2023.02.008

摘要/Abstract

摘要：

海量数据和强大算力的出现，促进了深度学习的发展，深度学习在智慧交通和医疗诊断等领域得到广泛应用，这给人们的日常生活带来许多便利。然而，机器学习中的隐私泄露问题也不容忽视。成员推理攻击通过推断数据样本是否用于训练机器学习模型，从而暴露用户训练数据的隐私。文章介绍了基于单阈值的成员推理攻击及特点，对不同攻击方法的成员和非成员的数据分布进行可视化，然后对成员推理攻击成功的内在机理进行分析，提出了基于双阈值函数的攻击模型，并通过实验对单阈值和双阈值的成员推理攻击进行系统性的分析对比，分析基于阈值成员推理攻击对不同模型和不同数据集的攻击表现。通过对多组控制变量的对比实验表明，基于双阈值函数的成员推理攻击在某些数据集和模型上，整体表现更加优异和稳定。

关键词: 深度学习, 成员推理攻击, 隐私泄露, 双阈值函数

Abstract:

The emergence of massive data and powerful computing power has brought deep learning to an unprecedented height, and its wide application in areas such as intelligent transportation and medical diagnosis has brought many conveniences to people’s daily lives. However, privacy leakage in machine learning cannot be ignored. Among them, the membership inference attack infers that whether the data sample can used in the training set of the machine learning model, thus interfering with the user’s training data. Firstly, this paper introduced the single-threshold-based membership inference attack and its characteristics, visualized the data distribution of members and non-members for different attack methods, then analyzed the internal mechanism of the successful membership inference attack, and proposed an attack model based on a double-threshold function, and systematically analyzed and compared single-threshold and double-threshold membership inference attacks through experiments, and analyzed the attack performance of threshold-based membership inference attacks on different models and different datasets. The comparative experiments on multiple groups of control variables show that the membership inference attack based on the double-threshold function has better performance on some data sets and models, and the overall performance is more stable.

Key words: deep learning, membership inference attack, privacy leak, double-threshold function

中图分类号:

TP309

陈得鹏, 刘肖, 崔杰, 仲红. 一种基于双阈值函数的成员推理攻击方法[J]. 信息网络安全, 2023, 23(2): 64-75.

CHEN Depeng, LIU Xiao, CUI Jie, ZHONG Hong. Research on Membership Inference Attack Method Based on Double Threshold Function[J]. Netinfo Security, 2023, 23(2): 64-75.

图/表 16

图1

图2

图3

图4

图5

表1

表2

表3

表4

表5

图6

图7

图8

图9

图10

图11

参考文献 16

[1]	SHOKRI R, STRONATI M, SONG Congzheng, et al. Membership Inference Attacks Against Machine Learning Models[C]// IEEE. Symposium on Security and Privacy. New York: IEEE, 2017: 3-18.
[2]	SALEM A, ZHANG Yang, HUMBERT M, et al. ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models[C]// The Internet Society. Network and Distributed System Security Symposium. New York: The Internet Society, 2019: 1-15.
[3]	LI Zheng, ZHANG Yang. Membership Leakage in Label-Only Exposures[C]// ACM. Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. New York: ACM, 2021: 880-895.
[4]	YEOM S, GIACOMELLI I, FREDRIKSON M, et al. Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting[C]// IEEE. 31st Computer Security Foundations Symposium. New York: IEEE, 2018: 268-282.
[5]	CHOO C C A, TRAMER F, CARLINI N, et al. Label-Only Membership Inference Attacks[C]// PMLR. Proceedings of the 38th International Conference on Machine Learning. New York: PMLR, 2021: 1964-1974.
[6]	HUI Bo, YANG Yuchen, YUAN Haolin, et al. Practical Blind Membership Inference Attack via Differential Comparisons[C]// The Internet Society. Network and Distributed System Security Symposium, New York: The Internet Society, 2021: 1-17.
[7]	TRUEX S, LIU Ling, GURSOY M E, et al. Demystifying Membership Inference Attacks in Machine Learning as a Service[J]. IEEE Transactions on Services Computing, 2019, 14(6): 2073-2089. doi: 10.1109/TSC.2019.2897554 URL
[8]	CHEN Jiale, ZHANG Jiale, ZHAO Yanchao, et al. Beyond Model-Level Membership Privacy Leakage: An Adversarial Approach in Federated Learning[C]// IEEE. 29th International Conference on Computer Communications and Networks. New York: IEEE, 2020: 1-9.
[9]	PICHLER G, ROMANELLI M, VEGA L R, et al. Perfectly Accurate Membership Inference by a Dishonest Central Server in Federated Learning[EB/OL]. (2022-03-30)[2022-10-18]. https://arxiv.org/ftp/arxiv/papers/2203/2203.16463.pdf.
[10]	HILPRECHT B, HRTERICH M, BERNAU D. Reconstruction and Membership Inference Attacks Against Generative Models[EB/OL]. (2019-06-07)[2022-10-18]. https://arxiv.org/pdf/1906.03006.pdf.
[11]	CHEN Dingfan, YU Ning, ZHANG Yang, et al. GAN-Leaks: A Taxonomy of Membership Inference Attacks Against Generative Models[C]// ACM. Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. New York: ACM, 2020: 343-362.
[12]	HU Hailong, PANG Jun. Membership Inference Attacks Against GANs by Leveraging Over-representation Regions[C]// ACM. Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security. New York: ACM, 2021: 2387-2389.
[13]	LIU Hongbin, JIA Jinyuan, QU Wenjie, et al. EncoderMI: Membership Inference Against Pre-Trained Encoders in Contrastive Learning[C]// ACM. Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security. New York: ACM, 2021: 2081-2095.
[14]	LI Jiacheng, LI Ninghui, RIBEIRO B. Membership Inference Attacks and Defenses in Classification Models[C]// ACM. The Eleventh ACM Conference on Data and Application Security and Privacy. New York: ACM, 2021: 5-16.
[15]	LONG Yunhui, BINDSCHAEDLER V, WANG Lei, et al. Understanding Membership Inferences on Well-Generalized Learning Models[EB/OL]. (2018-02-13)[2022-10-18]. https://arxiv.org/pdf/1802.04889.pdf.
[16]	REZAEI S, LIU Xin. On the Difficulty of Membership Inference Attacks[C]// IEEE. Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2021: 7892-7900.

环境及实验配置	对应值
CPU	Intel(R) Core(TM) i7-11700 @2.50 GHz(16 CPUs)
内存	16 GB
显卡	NVIDIA Tesla T4
数据集	CIFAR10、CIFAR100、STL10
目标模型结构	CNN、Vgg19、Resnet18

混淆矩阵		预测值
混淆矩阵		1	0
真实值	1	TP	FN
真实值	0	FP	TN

模型表现				基于阈值的成员推理攻击准确率
数据集	模型结构	训练准确度	测试准确度	${{A}_{\text{correct}}}$[4]	${{A}_{\text{max}}}$[2]	${{A}_{\text{entropy}}}$[2]	${{A}_{\text{double}}}$
CIFAR10	CNN	100%	77.55%	77.84%	74.84%	80.49%	75.88%
	VGG19	100%	92.69%	73.52%	73.64%	74.13%	74.30%
	Resnet18	100%	88.72%	74.65%	79.38%	76.53%	79.83%
CIFAR100	CNN	99.97%	43.74%	87.49%	79.23%	90.05%	81.74%
	VGG19	99.98%	71.46%	79.57%	79.74%	80.80%	79.73%
	Resnet18	99.98%	67.95%	80.57%	91.25%	89.41%	93.84%
STL10	CNN	100%	58.38%	71.24%	63.78%	69.22%	68.68%
	VGG19	100%	86.23%	56.92%	66.08%	56.94%	66.12%
	Resnet18	100%	77.50%	60.84%	70.74%	63.38%	70.90%
平均值	—	—	—	73.63%	75.41%	75.66%	76.78%

模型表现		基于阈值的成员推理攻击召回率
数据集	模型结构	${{A}_{\text{correct}}}$[4]	${{A}_{\text{max}}}$[2]	${{A}_{\text{entropy}}}$[2]	${{A}_{\text{double}}}$
CIFAR10	CNN	100%	95.78%	99.94%	97.57%
	VGG19	100%	98.26%	99.96%	98.38%
	Resnet18	100%	99.45%	100%	98.84%
CIFAR100	CNN	99.98%	95.25%	99.52%	95.93%
	VGG19	99.98%	98.21%	99.92%	98.44%
	Resnet18	99.98%	99.76%	99.93%	99.28%
STL10	CNN	100%	82.48%	100%	86.68%
	VGG19	100%	89.96%	100%	94.48%
	Resnet18	100%	93.28%	99.96%	92.92%

模型表现		基于阈值的成员推理攻击精确率和 F1分数
数据集	模型结构	${{A}_{\text{correct}}}$[4]		${{A}_{\max }}$[2]		${{A}_{\text{entropy}}}$[2]		${{A}_{\text{double}}}$
数据集	模型结构	精确率	F1 分数	精确率	F1 分数	精确率	F1 分数	精确率	F1 分数
CIFAR10	CNN	76.32%	0.866	74.85%	0.847	78.57%	0.879	76.93%	0.853
	VGG19	72.95%	0.844	74.17%	0.845	73.70%	0.847	74.11%	0.846
	Resnet18	73.81%	0.849	77.84%	0.873	75.27%	0.859	78.39%	0.874
CIFAR100	CNN	85.12%	0.920	79.66%	0.868	88.09%	0.935	81.05%	0.879
	VGG19	77.77%	0.874	78.70%	0.874	78.85%	0.881	78.59%	0.874
	Resnet18	78.63%	0.880	89.13%	0.942	89.41%	0.944	92.61%	0.958
STL10	CNN	63.48%	0.777	60.03%	0.695	61.90%	0.765	63.74%	0.738
	VGG19	53.72%	0.699	60.52%	0.726	53.73%	0.699	60.29%	0.737
	Resnet18	56.08%	0.719	64.30%	0.761	57.73%	0.732	64.58%	0.762