Small-Sample APT Attack Event Extraction Method Based on Large Model

doi:10.3969/j.issn.1671-1122.2025.09.002

Abstract

Abstract:

The detection and defense of APT attacks are relatively difficult. Automatically extracting APT attack events and key information from threat intelligence is of great significance for improving proactive defense capabilities and building high-quality threat intelligence. This capability enhances proactive defense strategies and supports the development of high-quality threat intelligence. However, threat intelligence related to APT often spans multiple attack stages and involves complex techniques with intricate semantics. Training accurate extraction models is hindered by the scarcity of high-quality datasets and limited sample sizes. This paper proposed a small-sample APT attack event extraction method based on large model. First, this method designed a data augmentation method for attack events based on large models. Using this method, the APTCNEE dataset and a Chinese corpus of APT attack events were created. Then, an ERNIE-BiLSTM-CRF model based on prompt learning was constructed. The experiment verifies the effectiveness of the method, with the F1 score higher than the baseline models, and data augmentation significantly boosts the performance of both trigger word and argument extraction.

Key words: large model, threat intelligence, event extraction, APT attack, data augmentation

CLC Number:

TP309

CAO Jun, XIANG Ga, REN Yawei, TAN Zicheng, YANG Qunsheng. Small-Sample APT Attack Event Extraction Method Based on Large Model[J]. Netinfo Security, 2025, 25(9): 1338-1347.

Figures/Tables 12

References 23

[1]	CHEN Ping, DESMET L, HUYGENS C. A Study on Advanced Persistent Threats[C]// Springer. IFIP International Conference on Communications and Multimedia Security. Heidelberg: Springer, 2014: 63-72.
[2]	LI Yuancheng, LUO Hao, WANG Qingle, et al. An Advanced Persistent Threat Model of New Power System Based on ATT&CK[J]. Netinfo Security, 2023, 23(2): 26-34.
	李元诚, 罗昊, 王庆乐, 等. 一种基于ATT&CK的新型电力系统APT攻击建模[J]. 信息网络安全, 2023, 23(2): 26-34.
[3]	GUO Zimeng, ZHU Guangjie, YANG Yijie, et al. Research on Railway Network Security Performance Based on APT Characteristics[J]. Netinfo Security, 2024, 24(5): 802-811.
	郭梓萌, 朱广劼, 杨轶杰, 等. 基于APT特征的铁路网络安全性能研究[J]. 信息网络安全, 2024, 24(5): 802-811.
[4]	ALSHAMRANI A, MYNENI S, CHOWDHARY A, et al. A Survey on Advanced Persistent Threats: Techniques, Solutions, Challenges, and Research Opportunities[J]. IEEE Communications Surveys & Tutorials, 2019, 21(2): 1851-1877.
[5]	ZHAO Xinqiang, FAN Bo, ZHANG Dongju. Research on APT Attack Defense System Based on Threat Discovery[J]. Netinfo Security, 2024, 24(7): 1122-1128.
	赵新强, 范博, 张东举. 基于威胁发现的APT攻击防御体系研究[J]. 信息网络安全, 2024, 24(7): 1122-1128.
[6]	WAGNER T D, MAHBUB K, PALOMAR E, et al. Cyber Threat Intelligence Sharing: Survey and Research Directions[EB/OL]. (2019-08-06)[2025-06-02]. https://doi.org/10.1016/j.cose.2019.101589.
[7]	LI Zhenyuan, ZENG Jun, CHEN Yan. AttacKG: Constructing Technique Knowledge Graph from Cyber Threat Intelligence Reports[C]// Springer. European Symposium on Research in Computer Security. Heidelberg: Springer, 2022: 589-609.
[8]	ZHANG Tongtao, JI Heng. Event Extraction with Generative Adversarial Imitation Learning[EB/OL]. (2018-04-21)[2025-06-02]. https://doi.org/10.48550/arXiv.1804.07881.
[9]	LUO Ning, DU Xiangyu, HE Yitong, et al. A Framework for Document-Level Cybersecurity Event Extraction from Open Source Data[C]// IEEE. IEEE 24th International Conference on Computer Supported Cooperative Work in Design(CSCWD). New York: IEEE, 2021: 422-427.
[10]	SATYAPANICH T, FERRARO F, FININ T. Casie: Extracting Cybersecurity Event Information from Text[J]. AAAI Technical Track: Natural Language Processing, 2020, 34(5): 8749-8757.
[11]	HUANG Zhiheng, XU Wei, YU Kai. Bidirectional LSTM-CRF Models for Sequence Tagging[EB/OL]. (2015-08-09)[2025-06-02]. https://arxiv.org/pdf/1508.01991.
[12]	JI Zhongxiang, WU Yue. Event Extraction of Chinese Text Based on Composite Neural Network[J]. Journal of Shanghai University(Natural Science), 2021, 27(3): 535-543.
	季忠祥, 吴悦. 基于组合神经网络的中文事件抽取[J]. 上海大学学报(自然科学版), 2021, 27(3): 535-543. doi: 10.12066/j.issn.1007-2861.2223
[13]	WANG Bo, WEI Wei, WU Yang, et al. Event Recognition in Chinese Emergencies Corpus Using ALBERT-BiLSTM-CRF[C]// IEEE. 2020 IEEE International Conference on Power, Intelligent Computing and Systems(ICPICS). New York: IEEE, 2020: 392-397.
[14]	XIANG Ga, SHI Chen, ZHANG Yangsen. An APT Event Extraction Method Based on BERT-BiGRU-CRF for APT Attack Detection[EB/OL]. (2023-08-04)[2025-06-02]. https://arxiv.org/pdf/1508.01991.
[15]	SUN Yu, WANG Shuohuan, FENG Shikun, et al. Ernie 3.0: Large-Scale Knowledge Enhanced Pre-Training for Language Understanding and Generation[EB/OL]. (2021-07-05)[2025-06-02]. https://doi.org/10.48550/arXiv.2107.02137.
[16]	GAO Yue, ZENG Chongqing, LIU Zhenye, et al. ERNIE-BC: An ERNIE-BiLSTM-CRF Fusion Model for Segmentation-Based Event Extraction[C]// IEEE. 2023 IEEE 9th International Conference on Cloud Computing and Intelligent Systems(CCIS). New York: IEEE, 2023: 424-428.
[17]	LIU Wanli, YONG Xinyou, CAO Kaichen, et al. Universal Information Extraction Method Based on Prompt Learning with ERNIE-BiLSTM-PN[J]. Journal of University of Electronic Science and Technology of China, 2025, 54(3): 411-423.
	刘万里, 雍新有, 曹开臣, 等. 基于提示学习的ERNIE-BiLSTM-PN通用信息抽取方法研究[J]. 电子科技大学学报, 2025, 54(3): 411-423.
[18]	WEI J, ZOU Kai. EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks[EB/OL]. (2019-08-25)[2025-06-02]. https://doi.org/10.48550/arXiv.1901.11196.
[19]	NAIR A R, SINGH R P, GUPTA D, et al. Evaluating the Impact of Text Data Augmentation on Text Classification Tasks Using DistilBERT[J]. Procedia Computer Science, 2024, 235: 102-111.
[20]	ZHANG Shaokang, RAN Ning. Contrastive Learning Based on Linguistic Knowledge and Adaptive Augmentation for Text Classification[EB/OL]. (2024-09-27)[2025-06-02]. https://doi.org/10.1016/j.knosys.2024.112189.
[21]	FENG Zijian, ZHOU Hanzhang, ZHU Zixiao, et al. Tailored Text Augmentation for Sentiment Analysis[EB/OL]. (2022-11-01)[2025-06-02]. https://doi.org/10.1016/j.eswa.2022.117605.
[22]	MENG Zihao, LIU Tao, ZHANG Heng, et al. CEAN: Contrastive Event Aggregation Network with LLM-Based Augmentation for Event Extraction[C]// ACL. The 18th Conference of the European Chapter of the Association for Computational Linguistics. Malta: ACL, 2024: 321-333.
[23]	YANG Lishan, FAN Xi, WANG Xiangyu, et al. Event Extraction Based on Self-Data Augmentation with Large Language Models[EB/OL]. (2025-01-31)[2025-06-02]. https://doi.org/10.1007/s12293-025-00436-8.

编号	类型	事件类型	论元1	论元2	论元3	论元4	论元5
1	攻击准备阶段	鱼叉攻击	伪造文件	真实文件	攻击者	受害目标	攻击战术
2		水坑攻击	伪造文件	真实文件	攻击者	受害目标	—
3		扫描	受害目标	—	—	—	—
4		窃取信息	受害目标	攻击者	被窃取目标	攻击武器	—
5	攻击实施阶段	特洛伊木马	攻击者	受害目标	攻击武器	攻击战术	—
6		蠕虫	攻击者	受害目标	攻击武器	—	—
7		漏洞利用	攻击者	受害目标	攻击武器	攻击战术	—
8		零日漏洞利用	攻击者	受害目标	攻击武器	—	—
9	持续攻击阶段	后门	攻击者	受害目标	攻击武器	—	—
10		病毒	攻击者	受害目标	攻击武器	攻击战术	—
11		凭据收集	攻击者	受害目标	攻击武器	攻击战术	—
12		远程访问木马	攻击者	受害目标	攻击武器	攻击战术	—
13		数据泄露	攻击者	受害目标	攻击武器	攻击战术	—

数据集	训练集/个	验证集/个	测试集/个	总计/个
APTCNEE	858	107	107	1072
AU_APTCNEE	4290	107	107	4504

数据集	训练集/个	验证集/个	测试集/个	总计/个
实例	11958	1498	3500	16956
事件	13478	1790	4372	19640
论元	29052	3696	8772	41520

模型	触发词识别			论元抽取
模型	Precision	Recall	F1	Precision	Recall	F1
BiLSTM-CRF	58.21%	49.68%	53.62%	56.17%	52.74%	54.39%
BERT-CRF	62.36%	52.87%	57.15%	60.54%	55.49%	57.89%
BERT-BiLSTM-CRF	64.78%	54.26%	59.05%	62.36%	56.47%	59.27%
APT-EBCEE	68.85%	59.35%	63.74%	65.17%	60.18%	62.53%

模型	触发词识别			论元抽取
模型	Precision	Recall	F1	Precision	Recall	F1
BiLSTM-CRF	60.21%	54.48%	57.21%	58.61%	55.18%	56.83%
BERT-CRF	64.23%	60.80%	62.45%	62.98%	58.64%	60.72%
BERT-BiLSTM-CRF	66.72%	61.25%	63.90%	64.19%	60.74%	62.41%
APT-EBCEE	71.34%	65.31%	68.26%	68.26%	64.18%	66.19%