Log Anomaly Detection Method Based on Improved Time Series Model

doi:10.3969/j.issn.1671-1122.2020.09.001

Abstract

Abstract:

Security log analysis plays an irreplaceable role in the field of network security. Aiming at the characteristics of security log, this paper proposes a multi-model combination time series anomaly detection algorithm. It combines the characteristics of time series, uses Fourier series to remove complex seasonal components, uses trend extrapolation to remove trend components, and then uses ESD testing to perform anomaly detection on random residual components. The experimental results show that the time series anomaly detection algorithm proposed in this paper has good detection accuracy.

Key words: security log, time series, anomaly detection

CLC Number:

TP309

LU Jiali. Log Anomaly Detection Method Based on Improved Time Series Model[J]. Netinfo Security, 2020, 20(9): 1-5.

Figures/Tables 4

References 12

[1]	HU Jiaojiao, WANG Xiaofeng, ZHANG Meng, et al. Time-series Data Anomaly Detection Method Based on Deep Learning[J]. Information and Control, 2019,48(1):1-8. doi: 10.1016/S0019-9958(81)90558-1 URL
	胡姣姣, 王晓峰, 张萌, 等. 基于深度学习的时间序列数据异常检测方法[J]. 信息与控制, 2019,48(1):1-8.
[2]	WANG Teng, JIAO Xuewei, GAO Yang. An Anomaly Detection Algorithm Based on Attention-GRU and iForest for Periodic Time Series[J]. Computer Engineering and Science, 2019,41(12):2217-2222.
	王腾, 焦学伟, 高阳. 一种基于Attention-GRU和iForest的周期性时间序列异常检测算法[J]. 计算机工程与科学, 2019,41(12):2217-2222.
[3]	MARTINS H, PALMA L, CARDOSO A, et al. A Support Vector Machine Based Technique for Online Detection of Outliers in Transient Time Series[EB/OL]. https://www.researchgate.net/publication/308106728_A_support_vector_machine_based_technique_for_online_detection_of_outliers_in_transient_time_series, 2020-5-15.
[4]	CHENG Yanyun, ZHANG Shouchao, YANG Yang. Research on Time Series Outlier Detection Based on Big Data[J]. Computer Technology and Development, 2016,26(5):139-144.
	程艳云, 张守超, 杨杨. 基于大数据的时间序列异常点检测研究[J]. 计算机技术与发展, 2016,26(5):139-144.
[5]	ZHANG Rixin, ZHU Yuelong, WAN Dingsheng, et al. Research on Two-stage Anomaly Detection Method Based on Feature Vector[J]. Information Technology, 2019,43(11):67-71, 77.
	张日新, 朱跃龙, 万定生, 等. 基于特征向量的两阶段异常检测方法研究[J]. 信息技术, 2019,43(11):67-71,77.
[6]	AYMEN A, ABDENNACEUR K, ADEL M. Outlier Detection for Wireless Sensor Networks Using Density-Based Clustering Approach[J]. Wireless Sensor Systems, 7(4):83-90.
[7]	SUN Jianshu, LOU Yuansheng, CHEN Yujun. Detection of Outliers in Hydrological Time Series Based on ARIMA-SVR[J]. Computer and Digital Engineering, 2018,46(2):225-230.
	孙建树, 娄渊胜, 陈裕俊. 基于ARIMA-SVR的水文时间序列异常值检测[J]. 计算机与数字工程, 2018,46(2):225-230.
[8]	CHEN Xingshu, JIANG Tianyu, ZENG Xuemei, et al. Network Anomaly Detection Based on Multidimensional Time Series Analysis[J]. Engineering Science and Technology, 2017,49(1):144-150.
	陈兴蜀, 江天宇, 曾雪梅, 等. 基于多维时间序列分析的网络异常检测[J]. 工程科学与技术, 2017,49(1):144-150.
[9]	PENA E H M, JUNIOR S B, RODRIGUES J J P C, et al. Anomaly Detection Using Digital Signature of Network Segment with Adaptive ARIMA Model and Paraconsistent Logic[EB/OL]. https://www.researchgate.net/publication/263542880_Anomaly_detection_using_digital_signature_of_network_segment_with_adaptive_ARIMA_model_and_Paraconsistent_Logic, 2020-5-15.
[10]	HOCHENBAUM J, VALLIS O S, KEJARIWAL A. Automatic Anomaly Detection in the Cloud Via Statistical Learning[EB/OL]. https://www.researchgate.net/publication/316471104_Automatic_Anomaly_Detection_in_the_Cloud_Via_Statistical_Learning, 2020-5-15.
[11]	MA Jiayu, HAN Zhaozhou. Simulation and Testing of Loss Simulation Model[J]. Statistics and Decision, 2017,17(6):27.
	马佳羽, 韩兆洲. 复杂季节时间序列模型研究[J]. 统计与决策, 2017,17(6):27.
[12]	ROSNER, BERNARD. Percentage Points for A Generalized ESD Many-Outlier Procedure[J]. Technometrics, 1983,25(2):165-172. doi: 10.1080/00401706.1983.10487848 URL

[1]	HUANG Na, HE Jingsha, WU Yabiao, LI Jianguo. Method of Insider Threat Detection Based on LSTM Regression Model [J]. Netinfo Security, 2020, 20(9): 17-21.
[2]	XU Yu, ZHOU You, LIN Lu, ZHANG Cong. Applied Research of Unsupervised Machine Learning in Game Anti-fraud [J]. Netinfo Security, 2020, 20(9): 32-36.
[3]	Wei WANG, Xudong SHEN. Research on Transfer Time Series Anomaly Detection Algorithm Based on Instance [J]. Netinfo Security, 2019, 19(3): 11-18.
[4]	Weichao YANG, Yuanbo GUO, Ya ZHONG, Shuaihui ZHEN. IoT Traffic Anomaly Detection Based on Device Type Identification and BP Neural Network [J]. Netinfo Security, 2019, 19(12): 53-63.
[5]	ZHU Haiqi, JIANG Feng. Research and Analysis of Anomaly Detection Technology for Operation and Maintenance Data in the Era of Artificial Intelligence [J]. Netinfo Security, 2019, 19(11): 24-35.
[6]	Hailian DENG, Yujing LIU, Yixuan GE, Jinshu SU. Research on Inter-domain Routing Anomaly Detection Technology [J]. Netinfo Security, 2019, 19(11): 63-70.
[7]	Xie LU, Shoushan LUO, Yumei ZHANG. Research on Hadoop-based Massive Security Log Clustering Algorithm [J]. Netinfo Security, 2018, 18(8): 56-63.
[8]	Wei LI, Xiaoxiao DI, Di WANG, Yunchun LI. Subgraph-based Network Behavior Models and Anomaly Detection for Server [J]. Netinfo Security, 2018, 18(2): 1-9.
[9]	Li HE, Yuanhui YAO. Detection and Recognition Strategy for Anomaly of Cloud Virtual Machine Based on Context Clustering [J]. Netinfo Security, 2018, 18(12): 54-65.
[10]	Ran AN, Xiaobo ZHU, Hanbing YAN. Research on a Method of Data Theft Detection Based on Time Series Decomposition [J]. Netinfo Security, 2017, 17(8): 76-82.
[11]	Gang ZHAO, Xingren YAO. Anomaly Detection Model Based on User Portrait [J]. Netinfo Security, 2017, 17(7): 18-24.
[12]	Xin WU, Yuesong YAN, Xiaoran LIU. Program Behavior Anomaly Detection Method Based on Improved HMM [J]. Netinfo Security, 2016, 16(9): 108-112.
[13]	Mingliang HE, Zemao CHEN, Jin ZUO. Cluster Anomaly Detection Algorithm Based on Multi-windows Mechanism [J]. Netinfo Security, 2016, 16(11): 33-39.
[14]	Jian TANG, Chun-lai SUN, Ke-feng MAO, Mei-ying JIA. Network Intrusion Anomaly Detection Model Based on Dimension Reduction Strategy Using Principal Component Analysis and Mutual Information [J]. Netinfo Security, 2015, 15(9): 78-83.
[15]	Ling-yun LI, Ji AO, Zhi QIAO, Jian LI. Research on Security Event Real-time Monitoring Framework Based on Micro-blog [J]. Netinfo Security, 2015, 15(1): 16-23.