基于自适应采样的机器遗忘方法

doi:10.3969/j.issn.1671-1122.2025.04.011

摘要/Abstract

摘要：

随着人工智能技术的快速发展，智能系统在医疗、工业等多个领域得到广泛应用。然而，智能系统中存储的大量用户数据一旦遭受恶意攻击，将对用户隐私构成严重威胁。为保护用户数据隐私，许多国家已出台相关法律法规，以确保用户享有“被遗忘权”。机器遗忘技术通常分为精确遗忘和近似遗忘两类，旨在通过调整模型参数，从已训练好的模型中消除特定数据的影响。精确遗忘方法利用剩余数据重新训练模型实现遗忘，但其计算成本较高；近似遗忘方法则通过少量参数更新实现遗忘，然而现有方法存在遗忘性能不足、遗忘时间过长等问题。文章提出一种基于自适应采样的机器遗忘方法，该方法先对模型训练过程中的梯度进行采样，随后利用少量梯度信息完成遗忘，具有广泛的适用性，可适配多种机器遗忘方法。实验结果表明，“先采样后遗忘”策略显著提升了近似遗忘性能，同时将精确遗忘时间减少了约22.9%，近似遗忘时间减少了约38.6%。

关键词: 机器遗忘, 隐私保护, 自适应采样, 被遗忘权

Abstract:

With the rapid development of artificial intelligence technologies, intelligent systems have been widely applied in various fields such as healthcare and industry. However, once a large amount of user data stored in intelligent systems is maliciously attacked, it will pose a serious threat to user privacy. To protect user data privacy, many countries have introduced relevant laws and regulations to ensure “the right to be forgotten”. Machine unlearning methods are typically divided into exact unlearning and approximate unlearning, aims to adjust model parameters to remove the influence of specific data from a trained model. Exact unlearning methods use the remaining data to retrain the model to achieve unlearning, but this approach is computationally expensive. Approximate unlearning methods use a smaller number of parameter updates to achieve unlearning, but existing approximate unlearning methods suffer from issues such as poor unlearning performance and long unlearning times. This paper proposed an adaptive sampling-based machine unlearning method, the method first sampled the gradients during the model training process, and then used a small amount of gradient information to complete unlearning. It had wide applicability and could be adapted to various machine forgetting methods. The experimental results show that the “sample first, unlearn later” approach can effectively improve the performance of approximate unlearning, while reducing the time for exact unlearning by about 22.9% and the time for approximate unlearning by about 38.6%.

Key words: machine unlearning, privacy protection, adaptive sampling, right to be forgotten

中图分类号:

TP309

何可, 王建华, 于丹, 陈永乐. 基于自适应采样的机器遗忘方法[J]. 信息网络安全, 2025, 25(4): 630-639.

HE Ke, WANG Jianhua, YU Dan, CHEN Yongle. Adaptive Sampling-Based Machine Unlearning Method[J]. Netinfo Security, 2025, 25(4): 630-639.

图/表 12

图1

表1

表2

表3

图2

图3

图4

表4

表5

表6

表7

表8

参考文献 29

[1]	TIAN Zhiyi, CUI Lei, LIANG Jie, et al. A Comprehensive Survey on Poisoning Attacks and Countermeasures in Machine Learning[J]. ACM Computing Surveys, 2023, 55(8): 1-35.
[2]	JONAS G, BAUERMEISTER H, DROGE H, et al. Inverting Gradients-How Easy is It to Break Privacy in Federated Learning[J]. Neural Information Processing Systems, 2020, 33: 16937-16947.
[3]	ZHAO Wei, REN Xiaoning, XUE Yinxing. Membership Inference Attacks Method Based on Ensemble Learning[J]. Netinfo Security, 2024, 24(8): 1252-1264.
	赵伟, 任潇宁, 薛吟兴. 基于集成学习的成员推理攻击方法[J]. 信息网络安全, 2024, 24(8): 1252-1264.
[4]	CHEN Guangke, ZHANG Yedi, SONG Fu. SLMIA-SR: Speaker-Level Membership Inference Attacks against Speaker Recognition Systems[EB/OL]. (2023-11-27)[2024-12-20]. https://arxiv.org/abs/2309.07983.
[5]	XIA Hui, QIAN Xiangyun. Invisible Backdoor Attack Based on Feature Space Similarity[J]. Netinfo Security, 2024, 24(8): 1163-1172.
	夏辉, 钱祥运. 基于特征空间相似的隐形后门攻击[J]. 信息网络安全, 2024, 24(8): 1163-1172.
[6]	LIU Gaoyang, WU Weiling, ZHANG Jinsheng, et al. Targeted Poisoning Attacks against Multimodal Contrastive Learning[J]. Netinfo Security, 2023, 23(11): 69-83.
	刘高扬, 吴伟玲, 张锦升, 等. 多模态对比学习中的靶向投毒攻击[J]. 信息网络安全, 2023, 23(11): 69-83.
[7]	VOIGT P, VON D B A. The EU General Data Protection Regulation (GDPR): A Practical Guide[M]. Heidelberg: Springer, 2024.
[8]	PARDAU S L. The California Consumer Privacy ACT: Towards a European-Style Privacy Regime in the United States[EB/OL]. (2018-06-28)[2024-12-20]. https://scholarship.law.ufl.edu/jtlp/vol23/iss1/2/.
[9]	WANG Weiqi, TIAN Zhiyi, ZHANG Chenhan, et al. Machine Unlearning: A Comprehensive Survey[EB/OL]. (2024-05-13)[2024-12-20]. https://arxiv.org/abs/2405.07406v2.
[10]	CHUNDAWAT V S, TARUN A K, MANDAL M, et al. Zero-Shot Machine Unlearning[J]. IEEE Transactions on Information Forensics and Security, 2023(18): 2345-2354.
[11]	CAO Yinzhi, YANG Junfeng. Towards Making Systems Forget with Machine Unlearning[C]// IEEE. 2015 IEEE Symposium on Security and Privacy. New York: IEEE, 2015: 463-480.
[12]	BOURTOULE L, CHANDRASEKARAN V, CHOQUETTE-CHOO C A, et al. Machine Unlearning[C]// IEEE. 2021 IEEE Symposium on Security and Privacy (SP). New York: IEEE, 2021: 141-159.
[13]	WARNECKE A, PIRCH L, WRESSNEGGER C, et al. Machine Unlearning of Features and Labels[EB/OL]. (2021-08-26)[2024-12-20]. https://arxiv.org/abs/2108.11577v4.
[14]	THUDI A, DEZA G, CHANDRASEKARAN V, et al. Unrolling SGD: Understanding Factors Influencing Machine Unlearning[C]// IEEE. 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P). New York: IEEE, 2022: 303-319.
[15]	BECKER A, LIEBIG T. Evaluating Machine Unlearning via Epistemic Uncertainty[EB/OL]. (2022-09-19)[2024-12-20]. https://arxiv.org/abs/2208.10836v2.
[16]	IZZO Z, SMART M A, CHAUDHURI K, et al. Approximate Data Deletion from Machine Learning Models[C]// PMLR. International Conference on Artificial Intelligence and Statistics. New York: PMLR, 2021: 2008-2016.
[17]	GRAVES L, NAGISETTY V, GANESH V. Amnesiac Machine Learning[EB/OL]. (2020-10-21)[2024-12-20]. https://doi.org/10.48550/arXiv.2010.10981.
[18]	BROPHY J, LOWD D. Machine Unlearning for Random Forests[C]// PMLR. International Conference on Machine Learning. New York: PMLR, 2021: 1092-1104.
[19]	GINART A, GUAN M Y, VALIANT G, et al. Making AI Forget You: Data Deletion in Machine Learning[J]. Neural Information Processing Systems, 2019, 316: 3518-3531.
[20]	MIRZASOLEIMAN B, KARBASI A, KRAUSE A. Deletion-Robust Submodular Maximization: Data Summarization with “ The Right to be Forgotten”[C]// ACM. The 34th International Conference on Machine Learning. New York: ACM, 2017: 2449-2458.
[21]	NGUYEN Q P, LOW B K H, JAILLET P. Variational Bayesian Unlearning[J]. Neural Information Processing Systems, 2020, 33: 16025-16036.
[22]	GUO Chuan, GOLDSTEIN T, HANNUN A, et al. Certified Data Removal from Machine Learning Models[C]// ACM. Proceedings of the 37th International Conference on Machine Learning (ICML’20). New York: ACM, 2020: 3832-3842.
[23]	MAHADEVAN A, MATHIOUDAKIS M. Certifiable Machine Unlearning for Linear Models[EB/OL]. (2021-06-29)[2024-12-20]. https://arxiv.org/abs/2106.15093v3.
[24]	SEKHARI A, ACHARYA J, KAMATH G, et al. Remember What You Want to Forget: Algorithms for Machine Unlearning[J]. Neural Information Processing Systems, 2021, 34: 18075-18086.
[25]	LIU Jinghan, RAM P, YAO Yuguang, et al. Model Sparsity Can Simplify Machine Unlearning[C]// ACM. Proceedings of the 37th International Conference on Neural Information Processing Systems(NIPS’23). New York: ACM, 2024, 2246: 51584-51605.
[26]	TARUN A K, CHUNDAWAT V S, MANDAL M, et al. Fast Yet Effective Machine Unlearning[J]. IEEE Transactions on Neural Networks and Learning Systems, 2024, 35(9): 13046-13055.
[27]	GOLATKAR A, ACHILLE A, SOATTO S. Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks[C]// IEEE. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE, 2020: 9304-9312.
[28]	ZHAO Yikai, ZHANG Yinda, LI Yuanpeng, et al. MinMax Sampling: A Near-Optimal Global Summary for Aggregation in the Wide Area[C]// ACM. Proceedings of the 2022 International Conference on Management of Data. New York: ACM, 2022: 744-758.
[29]	THUDI A, DEZA G, CHANDRASEKARAN V, et al. Unrolling SGD: Understanding Factors Influencing Machine Unlearning[C]// IEEE. 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P). New York: IEEE, 2022: 303-319.

名称	环境参数
操作系统	Ubuntu 20.04
GPU	24 GB NVIDIA GeForce RTX 4090
编程语言	Python 3.12
深度学习框架	PyTorch 2.5.0

方法	迭代次数/次	学习率
RE	100	0.01
FT	10	0.1
GA	5	0.0001

稀疏度	TA
0	92.89%
50%	92.51%
90%	92.28%
95%	92.13%
99%	89.88%

遗忘方法	运行时间/s
遗忘方法	采样后模型$M_{s}^{{}}$	原始模型$M$
RE	1023.28	1327.98
FT	90.48	146.65
GA	5.71	9.34

遗忘方法	FA
遗忘方法	$M$	0.9$M_{s}^{{}}$	0.95$M_{s}^{{}}$	0.99$M_{s}^{{}}$
RE	0	0	0	0
FT	5.93%	3.06%	1.28%	0.97%
GA	13.77%	9.19%	3.11%	2.22%