Intranet Log Anomaly Detection Model Based on Conformal Prediction

doi:10.3969/j.issn.1671-1122.2020.03.006

Abstract

Abstract:

Machine learning is the weakest link in cybersecurity threat detection systems. Evolving cybersecurity attacks exploit the conceptual drift of data to evade machine learning detection, causing detection models to degrade over time. In this paper, the statistical learning method of consistency metrics is used to alleviate the degradation problem of intranet security threat detection model based on log analysis. Compared with the static threshold-based detection method, the statistical learning method of consistency metric can dynamically adapt to the evolving security attack, perceive the conceptual drift of the underlying data, and alleviate the model degradation problem. This paper implements an internal network security detection model based on log analysis, effectively discovering the concept drift trend on the HDFS data set and alleviating the model degradation.

Key words: HDFS, anomly detection, conformal prediction, confusion matrix

CLC Number:

TP309

GU Zhaojun, REN Yitong, LIU Chunbo, WANG Zhi. Intranet Log Anomaly Detection Model Based on Conformal Prediction[J]. Netinfo Security, 2020, 20(3): 45-50.

Figures/Tables 8

References 18

[1]	VAARANDI R.A Data Clustering Algorithm for Mining Patterns from Event Logs[C]//IEEE. 3rd IEEE Workshop on IP Operations & Management, October 3, 2003, Kansas City, MO, USA. New Jersey: IEEE, 2003: 119-126.
[2]	VAARANDI R, PIHELGAS M.LogCluster-A Data Clustering and Pattern Mining Algorithm for Event Logs[C]//IEEE. 11th International Conference on Network and Service Management, November 9-13, 2015, Barcelona, Spain. New Jersey: IEEE, 2015: 1-7.
[3]	NAGAPPAN M, VOUK M A.Abstracting Log Lines to Log Event Types for Mining Software System Logs[C]//IEEE. 7th IEEE Working Conference on Mining Software Repositories, May 2-3 2010, Cape Town, South Africa. New Jersey: 2010: 114-117.
[4]	LIANG Tang, LI Tao, PERNG C S.LogSig: Generating System Events from Raw Textual Logs[C]//ACM. 20th ACM International Conference on Information and Knowledge Management, October 24-28, 2011, Glasgow, Scotland, UK. New York: ACM, 2011: 785-794.
[5]	HAMOONI H, DEBNATH B, XU J, et al.LogMine: Fast Pattern Recognition for Log Analytics[C]//ACM. 25th ACM International on Conference on Information and Knowledge Management, October 24-28, 2016, Indianapolis, IN, USA. ACM: New York: 2016: 1573-1582.
[6]	JIANG Zhenming, HASSAN A E, FLORA P, et al.Abstracting Execution Logs to Execution Events for Enterprise Applications(Short Paper)[C]//IEEE. 8th International Conference on Quality Software, August 12-13, 2008, Oxford, UK. New Jersey: IEEE, 2008: 181-186.
[7]	MAKANJU A, ZINCIRHEYWOOD A N, MILIOS E E.A Lightweight Algorithm for Message Type Extraction in System Application Logs[J]. IEEE Transactions on Knowledge and Data Engineering, 2012, 11(24): 1921-1936.
[8]	ZONG B, Song Qi, Min M R, et al. Deep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection[EB/OL]. , 2019-5-11.
[9]	ANGIULLI F, PIZZUTI C.Fast Outlier Detection in High Dimensional Spaces[C]//ACM. 6th European Conference on Principles of Data Mining and Knowledge Discovery, August 19-23, 2002, Helsinki, Finland. New York: ACM, 2002: 15-27.
[10]	ZIMEK A, SCHUBERT E, KRIEGEL H P.A Survey on Unsupervised Outlier Detection in High-dimensional Numerical Data[J]. Statistical Analysis and Data Mining: The ASA Data Science Journal, 2012, 5(5): 363-387.
[11]	PANG Guangsong, CAO Longbing, CHEN Ling, et al.Learning Representations of Ultrahigh-dimensional Data for Random Distance-based Outlier Detection[C]//ACM. 24th ACM SIGKDD International Conference on Knowledge Discovery and Data mining, August 19-23, 2018, London, UK. New York: ACM, 2018: 2041-2050.
[12]	BREUNIG M M, KRIEGEL H P, NG R T, et al.LOF: Identifying Density-based Local Outliers[C]//ACM. 2000 ACM SIGMOD International Conference on Management of Data, May 16-18, 2000, Dallas, Texas, USA. New York: ACM, 2000: 93-104.
[13]	RAMASWAMY S, RASTOGI R, SHIM K.Efficient Algorithms for Mining Outliers from Large Data Sets[J]. ACM SIGMOD Record, 2000, 29(2): 427-438.
[14]	DU Min, LI Feifei, ZHENG Guineng, et al.DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning[C]//ACM. 2017 ACM SIGSAC Conference, October 30-November 3, 2017, Dallas, TX, USA. New York: ACM, 2017: 1285-1298.
[15]	GAMA J, ŽLIOBAITĖ I, BIFET A, et al.A Survey on Concept Drift Adaptation[J]. ACM Computing Surveys, 2014, 46(4): 1-37.
[16]	KANTCHELIAN A, AFROZ S, HUANG L, et al.Approaches to Adversarial Drift[C]//ACM. ACM Workshop on Artificial Intelligence & Security, November 4, 2013, New York, USA. New York: ACM, 2013: 99-110.
[17]	THOMAS K, GRIER C, MA J, et al.Design and Evaluation of a Real-time URL Spam Filtering Service[C]//IEEE. 32nd IEEE Symposium on Security and Privacy, May 22-25, 2011, Berkeley, California, USA. New Jersey: IEEE, 2011: 447-462.
[18]	VOVK V, GAMMERMAN A, SHAFER, G. Algorithmic Learning in a Random World[EB/OL]. , 2019-6-12.

文本日志	结构化日志	block_id	事件id
PacketResponder 1 for block blk_16 terminating	PacketResponder <> for block <> terminating	16	E1
Verification succeeded for blk_490	Verification succeeded for <*>	490	E2
Verification succeeded for blk_16	Verification succeeded for <*>	16	E2

方法	基于阈值	基于p值
TP	113094	113151
TN	1616	1616
FP	59	59
FN	134	77

方法	基于阈值	基于p值
Accuracy	0.998	0.999
Precision	0.923	0.955
Recall	0.965	0.965
F1	0.944	0.960

方法	基于阈值	基于p值
TP	113228	113228
TN	1606	1664
FP	69	11
FN	0	0

方法	基于阈值	基于p值
Accuracy	0.999	1.0
Precision	1.0	1.0
Recall	0.959	0.993
F1	0.979	0.997