Adaptive Sampling-Based Machine Unlearning Method

doi:10.3969/j.issn.1671-1122.2025.04.011

Abstract

Abstract:

With the rapid development of artificial intelligence technologies, intelligent systems have been widely applied in various fields such as healthcare and industry. However, once a large amount of user data stored in intelligent systems is maliciously attacked, it will pose a serious threat to user privacy. To protect user data privacy, many countries have introduced relevant laws and regulations to ensure “the right to be forgotten”. Machine unlearning methods are typically divided into exact unlearning and approximate unlearning, aims to adjust model parameters to remove the influence of specific data from a trained model. Exact unlearning methods use the remaining data to retrain the model to achieve unlearning, but this approach is computationally expensive. Approximate unlearning methods use a smaller number of parameter updates to achieve unlearning, but existing approximate unlearning methods suffer from issues such as poor unlearning performance and long unlearning times. This paper proposed an adaptive sampling-based machine unlearning method, the method first sampled the gradients during the model training process, and then used a small amount of gradient information to complete unlearning. It had wide applicability and could be adapted to various machine forgetting methods. The experimental results show that the “sample first, unlearn later” approach can effectively improve the performance of approximate unlearning, while reducing the time for exact unlearning by about 22.9% and the time for approximate unlearning by about 38.6%.

Key words: machine unlearning, privacy protection, adaptive sampling, right to be forgotten

CLC Number:

TP309

HE Ke, WANG Jianhua, YU Dan, CHEN Yongle. Adaptive Sampling-Based Machine Unlearning Method[J]. Netinfo Security, 2025, 25(4): 630-639.

Figures/Tables 12

References 29

[1]	TIAN Zhiyi, CUI Lei, LIANG Jie, et al. A Comprehensive Survey on Poisoning Attacks and Countermeasures in Machine Learning[J]. ACM Computing Surveys, 2023, 55(8): 1-35.
[2]	JONAS G, BAUERMEISTER H, DROGE H, et al. Inverting Gradients-How Easy is It to Break Privacy in Federated Learning[J]. Neural Information Processing Systems, 2020, 33: 16937-16947.
[3]	ZHAO Wei, REN Xiaoning, XUE Yinxing. Membership Inference Attacks Method Based on Ensemble Learning[J]. Netinfo Security, 2024, 24(8): 1252-1264.
	赵伟, 任潇宁, 薛吟兴. 基于集成学习的成员推理攻击方法[J]. 信息网络安全, 2024, 24(8): 1252-1264.
[4]	CHEN Guangke, ZHANG Yedi, SONG Fu. SLMIA-SR: Speaker-Level Membership Inference Attacks against Speaker Recognition Systems[EB/OL]. (2023-11-27)[2024-12-20]. https://arxiv.org/abs/2309.07983.
[5]	XIA Hui, QIAN Xiangyun. Invisible Backdoor Attack Based on Feature Space Similarity[J]. Netinfo Security, 2024, 24(8): 1163-1172.
	夏辉, 钱祥运. 基于特征空间相似的隐形后门攻击[J]. 信息网络安全, 2024, 24(8): 1163-1172.
[6]	LIU Gaoyang, WU Weiling, ZHANG Jinsheng, et al. Targeted Poisoning Attacks against Multimodal Contrastive Learning[J]. Netinfo Security, 2023, 23(11): 69-83.
	刘高扬, 吴伟玲, 张锦升, 等. 多模态对比学习中的靶向投毒攻击[J]. 信息网络安全, 2023, 23(11): 69-83.
[7]	VOIGT P, VON D B A. The EU General Data Protection Regulation (GDPR): A Practical Guide[M]. Heidelberg: Springer, 2024.
[8]	PARDAU S L. The California Consumer Privacy ACT: Towards a European-Style Privacy Regime in the United States[EB/OL]. (2018-06-28)[2024-12-20]. https://scholarship.law.ufl.edu/jtlp/vol23/iss1/2/.
[9]	WANG Weiqi, TIAN Zhiyi, ZHANG Chenhan, et al. Machine Unlearning: A Comprehensive Survey[EB/OL]. (2024-05-13)[2024-12-20]. https://arxiv.org/abs/2405.07406v2.
[10]	CHUNDAWAT V S, TARUN A K, MANDAL M, et al. Zero-Shot Machine Unlearning[J]. IEEE Transactions on Information Forensics and Security, 2023(18): 2345-2354.
[11]	CAO Yinzhi, YANG Junfeng. Towards Making Systems Forget with Machine Unlearning[C]// IEEE. 2015 IEEE Symposium on Security and Privacy. New York: IEEE, 2015: 463-480.
[12]	BOURTOULE L, CHANDRASEKARAN V, CHOQUETTE-CHOO C A, et al. Machine Unlearning[C]// IEEE. 2021 IEEE Symposium on Security and Privacy (SP). New York: IEEE, 2021: 141-159.
[13]	WARNECKE A, PIRCH L, WRESSNEGGER C, et al. Machine Unlearning of Features and Labels[EB/OL]. (2021-08-26)[2024-12-20]. https://arxiv.org/abs/2108.11577v4.
[14]	THUDI A, DEZA G, CHANDRASEKARAN V, et al. Unrolling SGD: Understanding Factors Influencing Machine Unlearning[C]// IEEE. 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P). New York: IEEE, 2022: 303-319.
[15]	BECKER A, LIEBIG T. Evaluating Machine Unlearning via Epistemic Uncertainty[EB/OL]. (2022-09-19)[2024-12-20]. https://arxiv.org/abs/2208.10836v2.
[16]	IZZO Z, SMART M A, CHAUDHURI K, et al. Approximate Data Deletion from Machine Learning Models[C]// PMLR. International Conference on Artificial Intelligence and Statistics. New York: PMLR, 2021: 2008-2016.
[17]	GRAVES L, NAGISETTY V, GANESH V. Amnesiac Machine Learning[EB/OL]. (2020-10-21)[2024-12-20]. https://doi.org/10.48550/arXiv.2010.10981.
[18]	BROPHY J, LOWD D. Machine Unlearning for Random Forests[C]// PMLR. International Conference on Machine Learning. New York: PMLR, 2021: 1092-1104.
[19]	GINART A, GUAN M Y, VALIANT G, et al. Making AI Forget You: Data Deletion in Machine Learning[J]. Neural Information Processing Systems, 2019, 316: 3518-3531.
[20]	MIRZASOLEIMAN B, KARBASI A, KRAUSE A. Deletion-Robust Submodular Maximization: Data Summarization with “ The Right to be Forgotten”[C]// ACM. The 34th International Conference on Machine Learning. New York: ACM, 2017: 2449-2458.
[21]	NGUYEN Q P, LOW B K H, JAILLET P. Variational Bayesian Unlearning[J]. Neural Information Processing Systems, 2020, 33: 16025-16036.
[22]	GUO Chuan, GOLDSTEIN T, HANNUN A, et al. Certified Data Removal from Machine Learning Models[C]// ACM. Proceedings of the 37th International Conference on Machine Learning (ICML’20). New York: ACM, 2020: 3832-3842.
[23]	MAHADEVAN A, MATHIOUDAKIS M. Certifiable Machine Unlearning for Linear Models[EB/OL]. (2021-06-29)[2024-12-20]. https://arxiv.org/abs/2106.15093v3.
[24]	SEKHARI A, ACHARYA J, KAMATH G, et al. Remember What You Want to Forget: Algorithms for Machine Unlearning[J]. Neural Information Processing Systems, 2021, 34: 18075-18086.
[25]	LIU Jinghan, RAM P, YAO Yuguang, et al. Model Sparsity Can Simplify Machine Unlearning[C]// ACM. Proceedings of the 37th International Conference on Neural Information Processing Systems(NIPS’23). New York: ACM, 2024, 2246: 51584-51605.
[26]	TARUN A K, CHUNDAWAT V S, MANDAL M, et al. Fast Yet Effective Machine Unlearning[J]. IEEE Transactions on Neural Networks and Learning Systems, 2024, 35(9): 13046-13055.
[27]	GOLATKAR A, ACHILLE A, SOATTO S. Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks[C]// IEEE. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE, 2020: 9304-9312.
[28]	ZHAO Yikai, ZHANG Yinda, LI Yuanpeng, et al. MinMax Sampling: A Near-Optimal Global Summary for Aggregation in the Wide Area[C]// ACM. Proceedings of the 2022 International Conference on Management of Data. New York: ACM, 2022: 744-758.
[29]	THUDI A, DEZA G, CHANDRASEKARAN V, et al. Unrolling SGD: Understanding Factors Influencing Machine Unlearning[C]// IEEE. 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P). New York: IEEE, 2022: 303-319.

名称	环境参数
操作系统	Ubuntu 20.04
GPU	24 GB NVIDIA GeForce RTX 4090
编程语言	Python 3.12
深度学习框架	PyTorch 2.5.0

方法	迭代次数/次	学习率
RE	100	0.01
FT	10	0.1
GA	5	0.0001

稀疏度	TA
0	92.89%
50%	92.51%
90%	92.28%
95%	92.13%
99%	89.88%

遗忘方法	运行时间/s
遗忘方法	采样后模型$M_{s}^{{}}$	原始模型$M$
RE	1023.28	1327.98
FT	90.48	146.65
GA	5.71	9.34

遗忘方法	FA
遗忘方法	$M$	0.9$M_{s}^{{}}$	0.95$M_{s}^{{}}$	0.99$M_{s}^{{}}$
RE	0	0	0	0
FT	5.93%	3.06%	1.28%	0.97%
GA	13.77%	9.19%	3.11%	2.22%