Netinfo Security ›› 2023, Vol. 23 ›› Issue (2): 64-75.doi: 10.3969/j.issn.1671-1122.2023.02.008
Previous Articles Next Articles
CHEN Depeng, LIU Xiao, CUI Jie(), ZHONG Hong
Received:
2022-11-21
Online:
2023-02-10
Published:
2023-02-28
Contact:
CUI Jie
E-mail:cuijie@ahu.edu.cn
CLC Number:
CHEN Depeng, LIU Xiao, CUI Jie, ZHONG Hong. Research on Membership Inference Attack Method Based on Double Threshold Function[J]. Netinfo Security, 2023, 23(2): 64-75.
Add to citation manager EndNote|Ris|BibTeX
URL: http://netinfo-security.org/EN/10.3969/j.issn.1671-1122.2023.02.008
模型表现 | 基于阈值的成员推理攻击准确率 | ||||||
---|---|---|---|---|---|---|---|
数据集 | 模型 结构 | 训练 准确度 | 测试 准确度 | ${{A}_{\text{correct}}}$[ | ${{A}_{\text{max}}}$[ | ${{A}_{\text{entropy}}}$[ | ${{A}_{\text{double}}}$ |
CIFAR10 | CNN | 100% | 77.55% | 77.84% | 74.84% | 80.49% | 75.88% |
VGG19 | 100% | 92.69% | 73.52% | 73.64% | 74.13% | 74.30% | |
Resnet18 | 100% | 88.72% | 74.65% | 79.38% | 76.53% | 79.83% | |
CIFAR100 | CNN | 99.97% | 43.74% | 87.49% | 79.23% | 90.05% | 81.74% |
VGG19 | 99.98% | 71.46% | 79.57% | 79.74% | 80.80% | 79.73% | |
Resnet18 | 99.98% | 67.95% | 80.57% | 91.25% | 89.41% | 93.84% | |
STL10 | CNN | 100% | 58.38% | 71.24% | 63.78% | 69.22% | 68.68% |
VGG19 | 100% | 86.23% | 56.92% | 66.08% | 56.94% | 66.12% | |
Resnet18 | 100% | 77.50% | 60.84% | 70.74% | 63.38% | 70.90% | |
平均值 | — | — | — | 73.63% | 75.41% | 75.66% | 76.78% |
模型表现 | 基于阈值的成员推理攻击召回率 | ||||
---|---|---|---|---|---|
数据集 | 模型结构 | ${{A}_{\text{correct}}}$[ | ${{A}_{\text{max}}}$[ | ${{A}_{\text{entropy}}}$[ | ${{A}_{\text{double}}}$ |
CIFAR10 | CNN | 100% | 95.78% | 99.94% | 97.57% |
VGG19 | 100% | 98.26% | 99.96% | 98.38% | |
Resnet18 | 100% | 99.45% | 100% | 98.84% | |
CIFAR100 | CNN | 99.98% | 95.25% | 99.52% | 95.93% |
VGG19 | 99.98% | 98.21% | 99.92% | 98.44% | |
Resnet18 | 99.98% | 99.76% | 99.93% | 99.28% | |
STL10 | CNN | 100% | 82.48% | 100% | 86.68% |
VGG19 | 100% | 89.96% | 100% | 94.48% | |
Resnet18 | 100% | 93.28% | 99.96% | 92.92% |
模型表现 | 基于阈值的成员推理攻击精确率和 F1分数 | ||||||||
---|---|---|---|---|---|---|---|---|---|
数据集 | 模型 结构 | ${{A}_{\text{correct}}}$[ | ${{A}_{\max }}$[ | ${{A}_{\text{entropy}}}$[ | ${{A}_{\text{double}}}$ | ||||
精确率 | F1 分数 | 精确率 | F1 分数 | 精确率 | F1 分数 | 精确率 | F1 分数 | ||
CIFAR10 | CNN | 76.32% | 0.866 | 74.85% | 0.847 | 78.57% | 0.879 | 76.93% | 0.853 |
VGG19 | 72.95% | 0.844 | 74.17% | 0.845 | 73.70% | 0.847 | 74.11% | 0.846 | |
Resnet18 | 73.81% | 0.849 | 77.84% | 0.873 | 75.27% | 0.859 | 78.39% | 0.874 | |
CIFAR100 | CNN | 85.12% | 0.920 | 79.66% | 0.868 | 88.09% | 0.935 | 81.05% | 0.879 |
VGG19 | 77.77% | 0.874 | 78.70% | 0.874 | 78.85% | 0.881 | 78.59% | 0.874 | |
Resnet18 | 78.63% | 0.880 | 89.13% | 0.942 | 89.41% | 0.944 | 92.61% | 0.958 | |
STL10 | CNN | 63.48% | 0.777 | 60.03% | 0.695 | 61.90% | 0.765 | 63.74% | 0.738 |
VGG19 | 53.72% | 0.699 | 60.52% | 0.726 | 53.73% | 0.699 | 60.29% | 0.737 | |
Resnet18 | 56.08% | 0.719 | 64.30% | 0.761 | 57.73% | 0.732 | 64.58% | 0.762 |
[1] | SHOKRI R, STRONATI M, SONG Congzheng, et al. Membership Inference Attacks Against Machine Learning Models[C]// IEEE. Symposium on Security and Privacy. New York: IEEE, 2017: 3-18. |
[2] | SALEM A, ZHANG Yang, HUMBERT M, et al. ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models[C]// The Internet Society. Network and Distributed System Security Symposium. New York: The Internet Society, 2019: 1-15. |
[3] | LI Zheng, ZHANG Yang. Membership Leakage in Label-Only Exposures[C]// ACM. Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. New York: ACM, 2021: 880-895. |
[4] | YEOM S, GIACOMELLI I, FREDRIKSON M, et al. Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting[C]// IEEE. 31st Computer Security Foundations Symposium. New York: IEEE, 2018: 268-282. |
[5] | CHOO C C A, TRAMER F, CARLINI N, et al. Label-Only Membership Inference Attacks[C]// PMLR. Proceedings of the 38th International Conference on Machine Learning. New York: PMLR, 2021: 1964-1974. |
[6] | HUI Bo, YANG Yuchen, YUAN Haolin, et al. Practical Blind Membership Inference Attack via Differential Comparisons[C]// The Internet Society. Network and Distributed System Security Symposium, New York: The Internet Society, 2021: 1-17. |
[7] |
TRUEX S, LIU Ling, GURSOY M E, et al. Demystifying Membership Inference Attacks in Machine Learning as a Service[J]. IEEE Transactions on Services Computing, 2019, 14(6): 2073-2089.
doi: 10.1109/TSC.2019.2897554 URL |
[8] | CHEN Jiale, ZHANG Jiale, ZHAO Yanchao, et al. Beyond Model-Level Membership Privacy Leakage: An Adversarial Approach in Federated Learning[C]// IEEE. 29th International Conference on Computer Communications and Networks. New York: IEEE, 2020: 1-9. |
[9] | PICHLER G, ROMANELLI M, VEGA L R, et al. Perfectly Accurate Membership Inference by a Dishonest Central Server in Federated Learning[EB/OL]. (2022-03-30)[2022-10-18]. https://arxiv.org/ftp/arxiv/papers/2203/2203.16463.pdf. |
[10] | HILPRECHT B, HRTERICH M, BERNAU D. Reconstruction and Membership Inference Attacks Against Generative Models[EB/OL]. (2019-06-07)[2022-10-18]. https://arxiv.org/pdf/1906.03006.pdf. |
[11] | CHEN Dingfan, YU Ning, ZHANG Yang, et al. GAN-Leaks: A Taxonomy of Membership Inference Attacks Against Generative Models[C]// ACM. Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. New York: ACM, 2020: 343-362. |
[12] | HU Hailong, PANG Jun. Membership Inference Attacks Against GANs by Leveraging Over-representation Regions[C]// ACM. Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security. New York: ACM, 2021: 2387-2389. |
[13] | LIU Hongbin, JIA Jinyuan, QU Wenjie, et al. EncoderMI: Membership Inference Against Pre-Trained Encoders in Contrastive Learning[C]// ACM. Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security. New York: ACM, 2021: 2081-2095. |
[14] | LI Jiacheng, LI Ninghui, RIBEIRO B. Membership Inference Attacks and Defenses in Classification Models[C]// ACM. The Eleventh ACM Conference on Data and Application Security and Privacy. New York: ACM, 2021: 5-16. |
[15] | LONG Yunhui, BINDSCHAEDLER V, WANG Lei, et al. Understanding Membership Inferences on Well-Generalized Learning Models[EB/OL]. (2018-02-13)[2022-10-18]. https://arxiv.org/pdf/1802.04889.pdf. |
[16] | REZAEI S, LIU Xin. On the Difficulty of Membership Inference Attacks[C]// IEEE. Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2021: 7892-7900. |
[1] | TAN Liuyan, RUAN Shuhua, YANG Min, CHEN Xingshu. Educational Data Classification Based on Deep Learning [J]. Netinfo Security, 2023, 23(3): 96-102. |
[2] | XU Zhanyang, CHENG Luofei, CHENG Jianchun, XU Xiaolong. A Scheme of Optimizing Deep Learning Model Using Bi-ADMM [J]. Netinfo Security, 2023, 23(2): 54-63. |
[3] | JIA Fan, KANG Shuya, JIANG Weiqiang, WANG Guangtao. Vulnerability Similarity Algorithm Evaluation Based on NLP and Feature Fusion [J]. Netinfo Security, 2023, 23(1): 18-27. |
[4] | GAO Bo, CHEN Lin, YAN Yingjian. Research on Side Channel Attack Based on CNN-MGU [J]. Netinfo Security, 2022, 22(8): 55-63. |
[5] | ZHENG Yaohao, WANG Liming, YANG Jing. A Defense Method against Adversarial Attacks Based on Neural Architecture Search [J]. Netinfo Security, 2022, 22(3): 70-77. |
[6] | GUO Sensen, WANG Tongli, MU Dejun. Anomaly Detection Model Based on Generative Adversarial Network and Autoencoder [J]. Netinfo Security, 2022, 22(12): 7-15. |
[7] | ZHANG Zhi, LI Xin, YE Naifu, HU Kaixi. CAPTCHA Security Enhancement Method Incorporating Multiple Style Migration and Adversarial Examples [J]. Netinfo Security, 2022, 22(10): 129-135. |
[8] | LIU Shuo, ZHANG Xinglan. Intrusion Detection System Based on Dual Attention [J]. Netinfo Security, 2022, 22(1): 80-86. |
[9] | ZHU Xintong, TANG Yunqi, GENG Pengzhi. Detection Algorithm of Tamper and Deepfake Image Based on Feature Fusion [J]. Netinfo Security, 2021, 21(8): 70-81. |
[10] | LU Honglin, WANG Liming. User-oriented Data Privacy Preserving Method for Federated Learning that Supports User Disconnection [J]. Netinfo Security, 2021, 21(3): 64-71. |
[11] | MA Rui, CAI Manchun, PENG Shufan. A Deep Forgery Video Detection Model Based on Improved Xception Network [J]. Netinfo Security, 2021, 21(12): 109-117. |
[12] | PAN Xiaoqin, DU Yanhui. Forged Voice Identification Method Based on Feature Fusion and Multi-channel GRU [J]. Netinfo Security, 2021, 21(10): 1-7. |
[13] | XU Guotian, SHENG Zhenwei. DGA Malicious Domain Name Detection Method Based on Fusion of CNN and LSTM [J]. Netinfo Security, 2021, 21(10): 41-47. |
[14] | WU Jing, LU Tianliang, DU Yanhui. Generation of Malicious Domain Training Data Based on Improved Char-RNN Model [J]. Netinfo Security, 2020, 20(9): 6-11. |
[15] | WANG Wenhua, HAO Xin, LIU Yan, WANG Yang. The Safety Evaluation and Defense Reinforcement of the AI System [J]. Netinfo Security, 2020, 20(9): 87-91. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||