Vulnerability Similarity Algorithm Evaluation Based on NLP and Feature Fusion

doi:10.3969/j.issn.1671-1122.2023.01.003

Abstract

Abstract:

The study of vulnerability similarity helps security researchers to find solutions to new vulnerabilities from historical vulnerability information. The existing work on vulnerability similarity is not much, and the selection of its model is also lack of objective experimental data support. On this basis, this paper combined various word embedding technologies and deep learning auto-encoders to calculate semantic similarity from the perspective of vulnerability description text. At the same time, multi-dimensional feature data were extracted from public databases such as NVD, to calculate vulnerability feature similarity from the perspective of vulnerability features, and finally a dual angle vulnerability similarity measurement algorithm and evaluation scheme based on NLP and feature fusion was designed. Based on objective experimental analysis, the effects of various model combinations were compared from the aspects of numerical distribution, similarity discrimination, accuracy, etc. The final optimized model combination can obtain the highest F1 score of 0.927 in the determination of vulnerability similarity.

Key words: natural language processing, deep learning, vulnerability similarity, word embedding

CLC Number:

TP309

JIA Fan, KANG Shuya, JIANG Weiqiang, WANG Guangtao. Vulnerability Similarity Algorithm Evaluation Based on NLP and Feature Fusion[J]. Netinfo Security, 2023, 23(1): 18-27.

Figures/Tables 15

References 15

[1]	WANG Ju’an, ZHOU Linfeng, GUO Minzhe, et al. Measuring Similarity for Security Vulnerabilities[C]// IEEE. 43rd Hawaii International Conference on System Sciences. New York: IEEE, 2010: 1-10.
[2]	LE T H M, CHEN Huaming, ALI BABAR M. A Survey on Data-Driven Software Vulnerability Assessment and Prioritization[EB/OL]. (2021-07-25)[2022-01-10]. https://arxiv.org/abs/2107.08364.
[3]	XIA Zhiyang, YI Ping, YANG Tao. Static Vulnerability Detection Based on Neural Network and Code Similarity[J]. Computer Engineering, 2019, 45(12): 141-146.
	夏之阳, 易平, 杨涛. 基于神经网络与代码相似性的静态漏洞检测[J]. 计算机工程, 2019, 45(12):141-146.
[4]	LI Mei, GAO Qing, MA Sen, et al. Enhanced Simhash Algorithm for Code Similarity Detection[J]. Journal of Software, 2021, 32(7): 2242-2259.
	李玫, 高庆, 马森, 等. 面向代码相似性检测的相似哈希改进方法[J]. 软件学报, 2021, 32(7):2242-2259.
[5]	ZAHNG Dan, LUO Ping. Survey of Code Similarity Detection Methods and Tools[J]. Computer Science, 2020, 47(3): 5-10. doi: 10.11896/jsjkx.190500148
	张丹, 罗平. 代码相似性检测方法与工具综述[J]. 计算机科学, 2020, 47(3):5-10. doi: 10.11896/jsjkx.190500148
[6]	XIONG Hao, YAN Haihua, HUANG Yonggang, et al. Code Similarity Detection Approach Based on Back-Propagation Neural Network[J]. Computer Science, 2010, 37(3): 159-164.
	熊浩, 晏海华, 黄永刚, 等. 一种基于BP神经网络的代码相似性检测方法[J]. 计算机科学, 2010, 37(3):159-164.
[7]	KARLSSON L, BIDEH P N, HELL M. A Recommender System for User-Specific Vulnerability Scoring[D]. Lund: Lund University, 2019.
[8]	LI Zhen, ZOU Deqing, XU Shouhuai, et al. VulPecker: An Automated Vulnerability Detection System Based on Code Similarity Analysis[C]// ACM. 32nd Annual Conference on Computer Security Applications. New York: ACM, 2016: 201-213.
[9]	WANG Yisen, SHEN Jianjing, LIN Jian, et al. Staged Method of Code Similarity Analysis for Firmware Vulnerability Detection[J]. IEEE Access, 2019, 7: 14171-14185. doi: 10.1109/ACCESS.2019.2893733
[10]	WANG Chunliu, YANG Yonghui, DENG Fei, et al. A Review of Text Similarity Approaches[J]. Information Science, 2019, 37(3): 158-168.
	王春柳, 杨永辉, 邓霏, 等. 文本相似度计算方法研究综述[J]. 情报科学, 2019, 37(3):158-168.
[11]	LI Hailin, ZOU Jinchuan. Text Similarity Measure Method Based on Classified Dictionary[J]. CAAI Transactions on Intelligent Systems, 2017, 12(4): 556-562.
	李海林, 邹金串. 基于分类词典的文本相似性度量方法[J]. 智能系统学报, 2017, 12(4):556-562.
[12]	YI Jin, WU Runpu, LI Juan, et al. Similar Vulnerability Query Based on Text Mining[C]// IEEE. 11th International Symposium on Communications & Information Technologies(ISCIT). New York:IEEE, 2011: 339-342.
[13]	XU Jianghong, SHI Yijie. Research on the Similarity Measurement Method of Security Vulnerabilities Based on Vulnerability Features[EB/OL]. (2021-03-18)[2021-11-10]. http://www.paper.edu.cn/releasepaper/content/202103-188.
	徐江洪, 时忆杰. 基于漏洞特征的安全漏洞相似性度量方法的研究[EB/OL]. (2021-03-18)[2021-11-10]. http://www.paper.edu.cn/releasepaper/content/202103-188.
[14]	National Vulnerability Database(NVD). NVD CWE Slice[EB/OL]. [2021-11-10]. https://nvd.nist.gov/vuln/categories.
	国家漏洞数据库(NVD). CWE层级结构关系[EB/OL]. [2021-11-10]. https://nvd.nist.gov/vuln/categories.
[15]	YU Tingting, XU Pengna, JIANG Yu’e, et al. Text Similarity Method Based on the Improved Jaccard Coefficient[J]. Computer Systems & Applications, 2017, 26(12): 137-142.
	俞婷婷, 徐彭娜, 江育娥, 等. 基于改进的Jaccard系数文档相似度计算方法[J]. 计算机系统应用, 2017, 26(12):137-142.

基本指标	指标类别	赋分
攻击途径AV	本地	0.395
	相邻网络	0.646
	网络	1
攻击复杂度AC	高	0.35
	中	0.61
	低	0.71
认证AU	多次	0.45
	单次	0.56
	不需要	0.704
机密性CI	无	0
	部分	0.275
	完全	0.66
完整性II	无	0
	部分	0.275
	完全	0.66
可用性AI	无	0
	部分	0.275
	完全	0.66

数据库	严重性等级	严重性分数	赋分
CNNVD	超危	7.97	0.797
	高危	6.51	0.651
	中危	5.16	0.516
	低危	2.80	0.280
CNVD	高	6.81	0.681
	中	5.21	0.521
	低	3.66	0.366

特征类别	权重分配	特征	权重
CVSSv2漏洞基本指标	0.12	攻击途径（AV）	0.04
		攻击复杂度（AC）	0.04
		认证（AU）	0.04
	0.18	机密性（CI）	0.06
		完整性（II）	0.06
		可用性（AI）	0.06
CVSSv2漏洞基本评分	0.1	基本分数（BaseScore）	0.05
		影响力分数（Impact）	0.025
		可利用性分数（Exploitability）	0.025
漏洞严重性特征	0.1	CNVD严重性分数（CNVD_SecurityScore）	0.05
漏洞严重性特征	0.1	CNNVD严重性分数（CNNVD_SecurityScore）	0.05
受影响的产品集	0.2	产品	0.2
漏洞类型	0.3	类型	0.3
合计	1	13	1

	TF-IDF	TF-IDF+ LSTM	TF-IDF+ GRU	TF-IDF+ BiLSTM	TF-IDF+ BiGRU
A	0.095	0.101	0.101	0.101	0.101
B	0.025	0.029	0.029	0.029	0.029
C	0.145	0.155	0.155	0.155	0.155
	GloVe	GloVe+ LSTM	GloVe+ GRU	GloVe+ BiLSTM	GloVe+ BiGRU
A	0.197	0.234	0.236	0.210	0.219
B	-0.007	0.080	0.091	0.046	0.045
C	0.347	0.353	0.347	0.344	0.352
	Word2Vec	Word2Vec+ LSTM	Word2Vec+ GRU	Word2Vec+ BiLSTM	Word2Vec+ BiGRU
A	0.179	0.028	0.014	0.040	0.042
B	-0.002	-0.001	-0.002	0.001	0.003
C	0.320	0.051	0.026	0.069	0.075
	FastText	FastText+ LSTM	FastText+ GRU	FastText+ BiLSTM	FastText+ BiGRU
A	0.187	0.028	0.017	0.044	0.040
B	-0.005	-0.002	-0.001	0.000	-0.002
C	0.333	0.050	0.030	0.078	0.072

	TF-IDF	TF-IDF+ LSTM	TF-IDF+ GRU	TF-IDF+ BiLSTM	TF-IDF+ BiGRU
A	0.63	0.66	0.66	0.66	0.66
B	0.58	0.58	0.58	0.58	0.58
	GloVe	GloVe+LSTM	GloVe+ GRU	GloVe+ BiLSTM	GloVe+ BiGRU
A	0.77	0.77	0.79	0.76	0.79
B	0.73	0.71	0.73	0.71	0.73
	Word2Vec	Word2Vec+ LSTM	Word2Vec+ GRU	Word2Vec+ BiLSTM	Word2Vec+ BiGRU
A	0.76	0.75	0.66	0.70	0.70
B	0.72	0.55	0.50	0.59	0.58
	FastText	FastText+ LSTM	FastText+ GRU	FastText+ BiLSTM	FastText+ BiGRU
A	0.76	0.71	0.67	0.71	0.70
B	0.72	0.54	0.52	0.57	0.54