Abnormal Traffic Detection Technology Based on Data Augmentation and Model Update

doi:10.3969/j.issn.1671-1122.2020.02.009

Abstract

Abstract:

Due to the endless network attack means, the data samples are constantly changing, resulting in low accuracy of anomaly detection. The traditional network abnormal traffic detection method is detected by rule matching. The detection method is relatively simple, and it is difficult to adapt to a complex and flexible large-scale network environment. To this end, this paper proposes an abnormal traffic detection technology based on data augmentation and model update. In order to solve the problem of data imbalance, this paper introduces the SMOTE algorithm to oversample the minority samples, and removes the noise data with the ENN algorithm. The important features are extracted by the random forest algorithm, and the model update is implemented with the feature importance as the distance metric in the improved KNN algorithm. Finally, the CatBoost classification algorithm is used to classify network traffic data. In the model iterative update process, the detection of abnormal traffic is better. Compared with HCPTC-IDS, the detection accuracy and false positive rate are improved. The experimental results on the KDD 99 dataset show that the multi-classification detection accuracy of this model is as high as 96.52%, and the false positive rate is only 0.92%.

Key words: network abnormal traffic detection, data imbalance, character importance, model updating, KDD 99

CLC Number:

TP309

ZHANG Hao, CHEN Long, WEI Zhiqiang. Abnormal Traffic Detection Technology Based on Data Augmentation and Model Update[J]. Netinfo Security, 2020, 20(2): 66-74.

Figures/Tables 16

References 15

[1]	Network Security Law of the People’s Republic of China[EB/OL]. , 2016-11-7.
	中华人民共和国网络安全法[EB/OL]. , 2016- 11-7.
[2]	BELOUCH M, EL S, IDHAMMAD M.A Two-Stage Classifier Approach Using RepTree Algorithm for Network Intrusion Detection[J]. International Journal of Advanced Computer Science and Applications, 2017, 8(6): 137-142.
[3]	JIA Fan, YAN Yan, ZHANG Jiaqi.k-means Based Feature Reduction for Network Anomaly Detection[J]. Journal of Tsinghua University(Science and Technology), 2018, 58(2): 137-142.
	贾凡,严妍,张家琪.基于k-means聚类特征消减的网络异常检测[J]. 清华大学学报(自然科学版),2018,58(2):137-142.
[4]	ASHFAQ R A R, WANG Xizhao, HUANG Zhexue, et al. Fuzziness Based Semi-Supervised Learning Approach for Intrusion Detection System[J]. Information Sciences, 2016, 378(C): 484-497.
[5]	AHMIM A, DERDOUR M, FERRAG M A.An Intrusion Detection System Based on Combining Probability Predictions of a Tree of Classifiers[J]. International Journal of Communication Systems, 2018, 31(9): e3547.
[6]	YANG Xudong, GAO Ling, WANG Hai, et al.A Cooperative Deep Belief Network for Intrusion Detection[C]//IEEE. 2018 Sixth International Conference on Advanced Cloud and Big Data, August 12-15, Lanzhou, China. New York: IEEE, 2018: 230-236.
[7]	GREGGIO N.Anomaly Detection in IDSs by Means of Unsupervised Greedy Learning of Finite Mixture Models[J]. Soft Computing, 2018, 22(10): 3357-3372.
[8]	CHAWLA N V, BOWYER K W, HALL L O, et al.SMOTE: Synthetic Minority Over-Sampling Technique[J]. Journal of Artificial Intelligence Research, 2002, 16(1): 321-357.
[9]	WILSON D L. Asymptotic Properties of Nearest Neighbor Rules Using Edited Data[J]. IEEE Transactions on Systems, Man and Cybernetics, 1972, SMC-2(3): 408-421.
[10]	BREIMAN L.Random Forests[J]. Machine Learning, 2001, 45(1): 5-32.
[11]	SYLVESTER E V A, BENTZEN P, BRADBURY I R, et al. Applications of Random Forest Feature Selection for Fine-Scale Genetic Population Assignment[J]. Evolutionary Applications, 2018, 11(2): 153-165.
[12]	PROKHORENKOVA L, GUSEV G, VOROBEV A, et al.CatBoost: Unbiased Boosting with Categorical Features[J]. Advances in Neural Information Processing Systems, 2018, 31(3): 6638-6648.
[13]	HART P E.The Condensed Nearest Neighbor Rule[J]. IEEE Transactions on Information Theory, 1968, 14(3): 515-516.
[14]	COHEN W W.Fast Effective Rule Induction[C]//ACM. Proceedings of the Twelfth International Conference on Machine Learning Machine Learning, July 9-12, 1995, Tahoe City, California. New York: ELSEVISER, 1995: 115-123.
[15]	JORDAN M I, BISHOP C.Neural Networks[J]. ACM Computing Surveys, 1996, 28(1): 73-75.

类别	KDD 99中10%训练集数据		KDD 99中10%测试集数据
类别	所有	去冗余	所有	去冗余
Normal	97278	87832	60593	47913
DoS	391458	54572	229853	23568
Probe	4107	2130	4166	2678
R2L	1126	999	16189	2913
U2R	52	52	228	215
Total	494021	145585	311029	77287

类型	Normal	DoS	Probe	R2L	U2R	Total
训练数据集	12000	24819	2130	999	52	40000
测试数据集	47913	23568	2678	2913	215	77287

特征	重要性	特征	重要性
dst_host_srv_count	0.1026	diff_srv_rate	0.0349
dst_bytes	0.0905	dst_host_same_srv_rate	0.0321
service	0.0828	dst_host_count	0.0303
logged_in	0.0750	flag	0.0284
dst_host_diff_srv_rate	0.0555	protocol_type	0.0271
count	0.0526	num_compromised	0.0269
srv_count	0.0483	num_file_creations	0.0265
serror_rate	0.0475	dst_host_srv_diff_host_rate	0.0238
dst_host_same_src_port_rate	0.0399	hot	0.0220
same_srv_rate	0.0361	duration	0.0181

类别	Normal	DoS	Probe	R2L	U2R	Total	类型
Num	12000	24819	2130	999	52	40000	Training
Num	24784	24801	24784	24799	24819	123987	SMOTE-ENN

		预测类型
		负类	正类
实际类型	负类	TN	FP
实际类型	正类	FN	TP