基于实例的迁移时间序列异常检测算法研究

doi:10.3969/j.issn.1671-1122.2019.03.002

信息网络安全 ›› 2019, Vol. 19 ›› Issue (3): 11-18.doi: 10.3969/j.issn.1671-1122.2019.03.002

基于实例的迁移时间序列异常检测算法研究

王伟^1,², 沈旭东^1,²

1. 同济大学电子与信息工程学院,上海 200092
2. 同济大学嵌入式系统与服务计算教育部重点实验室,上海 200092

收稿日期:2018-12-25 出版日期:2019-03-19 发布日期:2020-05-11
作者简介:
作者简介：王伟（1979—）,男,湖北,副教授,博士,主要研究方向为信息安全、并行分布式计算;沈旭东（1994—）,男,浙江,硕士研究生,主要研究方向为大数据、时间序列分析。
基金资助:
国家自然科学基金面上项目[61672384]

Research on Transfer Time Series Anomaly Detection Algorithm Based on Instance

Wei WANG^1,², Xudong SHEN^1,²

1. College of Electronics and Information Engineering, Tongji University, Shanghai 200092, China
2. The Key Laboratory of Embedded System and Service Computing of Ministry of Education, Tongji University, Shanghai 200092, China

Received:2018-12-25 Online:2019-03-19 Published:2020-05-11

摘要/Abstract

摘要：

时间序列异常检测不管在学术界还是工业界都正引起人们极大的兴趣,但同时也存在异常标签数据缺失严重这一普遍问题。为了解决该问题,文章提出了基于实例的迁移时间序列异常检测算法——InsTransAnomalyDetect算法。该算法通过构建有效的决策函数来迁移实例,将原来的无监督异常检测任务转化为监督学习的任务。算法集成两种决策函数,分别是基于密度的决策函数和基于聚类的决策函数。文章最后将该方法与两种经典的异常检测算法在24个数据集上进行效果对比。实验结果表明,在24个数据集中,文中算法的表现优于无监督的异常检测算法的数据集有21个,平均准确率提升20%左右。实验证明了文中算法的优越性。

关键词: 异常检测, 时间序列, 迁移学习

Abstract:

Time series anomaly detection is attracting great interest both in academia and industry. A common and ubiquitous problem is the lack of abnormal tag data. In order to solve this problem, this paper proposes InsTransAnomalyDetect which is a time series anomaly detection algorithm based on transfer learning. This algorithm transforms the instance by constructing an effective transfer decision function. According to the decision function, this algorithm transforms the original unsupervised anomaly detection task into a supervised learning task. The algorithm integrates two decision functions, which are density-based decision function and cluster-based decision function. Finally, the method is compared with two classical anomaly detection algorithms on 24 data sets. The experimental results show that 21 of the 24 data sets outperform the unsupervised anomaly detection algorithm, and the average accuracy rate is about 20% better. Experiments show that the migration learning method is promising and proves the superiority of the algorithm framework.

Key words: anomaly detection, time series, transfer learning

中图分类号:

TP309

王伟, 沈旭东. 基于实例的迁移时间序列异常检测算法研究[J]. 信息网络安全, 2019, 19(3): 11-18.

Wei WANG, Xudong SHEN. Research on Transfer Time Series Anomaly Detection Algorithm Based on Instance[J]. Netinfo Security, 2019, 19(3): 11-18.

图/表 4

图1

图2

图3

表1

参考文献 14

[1]	PAN S J, YANG Qiang .A Survey on Transfer Learning[J]. IEEE Transactions on Knowledge & Data Engineering, 2010, 22(10): 1345-1359.
[2]	WEISS K, KHOSHGOFTAAR T M, WANG Dingding.A Survey of Transfer Learning[J]. Journal of Big Data, 2016, 3(1): 3-9.
[3]	IGNATOVA I A, POKIDYSHEVA L I, SMIRNOVA S V.Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods, and Techniques[M]. Hershey: IGI Global, 2009.
[4]	CHATTOPADHYAY R, YE Jieping, PANCHANATHAN S, et al.Multi-source Domain Adaptation and Its Application to Early Detection of Fatigue[C]//ACM. 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, August 21-24, 2011, San Diego, California, USA. New York: ACM, 2011: 717-725.
[5]	YAO Yi, DORETTO G.Boosting for Transfer Learning with Multiple Sources[C]//IEEE. 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, June 13-18, 2010, San Francisco, CA, USA. New Jersey: IEEE, 2010: 1855-1862.
[6]	XIA Rui, ZONG Chengqing, HU Xuelei, et al.Feature Ensemble Plus Sample Selection: Domain Adaptation for Sentiment Classification[J]. IEEE Intelligent Systems, 2013, 28(3): 10-18.
[7]	SPIEGEL S. Transfer Learning for Time Series Classification in Dissimilarity Spaces[EB/OL]., 2018-11-21.
[8]	FAWAZ H I,FORESTIER G, WEBER J, et al. Transfer Learning for Time Series Classification[EB/OL]. .
[9]	ANDREWS J T, TANAY T, MORTON E, et al. Transfer Representation-learning for Anomaly Detection[EB/OL]. .
[10]	AN Ran, ZHU Xiaobo, YAN Hanbing.Study on Data Stealth Event Detection Method Based on Time Series Decomposition[J]. Netinfo Security, 2017, 17(8): 76-82.
	安冉,朱小波,严寒冰. 一种基于时间序列分解的数据窃密事件检测方法研究[J]. 信息网络安全,2017, 17(8):76-82.
[11]	LAPTEV N, AMIZADEH S, FLINT I.Generic and Scalable Framework for Automated Time-series Anomaly Detection[C]//ACM. 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, August 10-13, 2015, Sydney, NSW, Australia. New York: ACM, 2015: 1939-1947.
[12]	MA Minghua, ZHANG Shenglin, DAN Pei, et al.Robust and Rapid Adaption for Concept Drift in Software System Anomaly Detection[C]//IEEE. 29th International Symposium on Software Reliability Engineering, October 15-18, 2018, Memphis, TN, USA. New Jersey: IEEE, 2018: 13-24.
[13]	KHA ,NH, ANH D T. From Cluster-based Outlier Detection to Time Series Discord and Applications in Knowledge Discovery and Data Mining[EB/OL]..1007%2F978-3-319-25660-3_2, 2018-11-21.
[14]	LIU F T, KAI M T, ZHOU Zhihua.Isolation Forest[C]//IEEE. 8th IEEE International Conference on Data Mining, December 15-19, 2008, Pisa, Italy. New Jersey: IEEE, 2008: 413-422.

源域	目标域	ClusterBased-ITAD		DensityBased-ITAD
源域	目标域	迁移数量	结果	迁移数量	结果
01:00	00:00	14	89%	13	89%
03:00	00:00	11	79%	11	74%
21:00	00:00	10	79%	9	58%
combo	00:00	60	90%	46	85%
03:00	06:00	6	52%	5	52%
11:00	06:00	15	56%	8	56%
21:00	06:00	7	52%	8	48%
combo	06:00	79	57%	54	44%
03:00	15:00	6	58%	6	58%
11:00	15:00	19	65%	10	58%
21:00	15:00	7	56%	9	58%
combo	15:00	85	67%	54	58%
03:00	19:00	7	52%	5	52%
11:00	19:00	15	60%	8	49%
21:00	19:00	9	52%	8	45%
combo	19:00	81	56%	50	50%
源域	目标域	CBLOF		IsolationForest
源域	目标域	结果		结果
01:00	00:00	58%		61%
03:00	00:00	52%		55%
21:00	00:00	52%		56%
combo	00:00	63%		63%
03:00	06:00	39%		41%
11:00	06:00	35%		38%
21:00	06:00	39%		41%
combo	06:00	35%		38%
03:00	15:00	23%		22%
11:00	15:00	30%		31%
21:00	15:00	19%		19%
combo	15:00	27%		30%
03:00	19:00	44%		46%
11:00	19:00	40%		41%
21:00	19:00	40%		41%
combo	19:00	44%		46%

[1]	郭春, 陈长青, 申国伟, 蒋朝惠. 一种基于可视化的勒索软件分类方法[J]. 信息网络安全, 2020, 20(4): 31-39.
[2]	顾兆军, 任怡彤, 刘春波, 王志. 基于一致性预测算法的内网日志检测模型[J]. 信息网络安全, 2020, 20(3): 45-50.
[3]	杨威超, 郭渊博, 钟雅, 甄帅辉. 基于设备型号分类和BP神经网络的物联网流量异常检测[J]. 信息网络安全, 2019, 19(12): 53-63.
[4]	朱海麒, 姜峰. 人工智能时代面向运维数据的异常检测技术研究与分析[J]. 信息网络安全, 2019, 19(11): 24-35.
[5]	邓海莲, 刘宇靖, 葛一漩, 苏金树. 域间路由异常检测技术研究[J]. 信息网络安全, 2019, 19(11): 63-70.
[6]	李巍, 狄晓晓, 王迪, 李云春. 基于子图的服务器网络行为建模及异常检测方法研究[J]. 信息网络安全, 2018, 18(2): 1-9.
[7]	何利, 姚元辉. 基于上下文聚类的云虚拟机异常检测与识别策略[J]. 信息网络安全, 2018, 18(12): 54-65.
[8]	安冉, 朱小波, 严寒冰. 一种基于时间序列分解的数据窃密事件检测方法研究[J]. 信息网络安全, 2017, 17(8): 76-82.
[9]	赵刚, 姚兴仁. 基于用户画像的异常行为检测模型[J]. 信息网络安全, 2017, 17(7): 18-24.
[10]	吴鑫, 严岳松, 刘晓然. 基于改进HMM的程序行为异常检测方法[J]. 信息网络安全, 2016, 16(9): 108-112.
[11]	杨连群, 温晋英, 刘树发, 王峰. 一种改进的图分割算法在用户行为异常检测中的应用[J]. 信息网络安全, 2016, 16(6): 35-40.
[12]	何明亮, 陈泽茂, 左进. 基于多窗口机制的聚类异常检测算法[J]. 信息网络安全, 2016, 16(11): 33-39.
[13]	刘汝隽, 辛阳. 网络安全数据可视分析系统的设计与实现[J]. 信息网络安全, 2016, 16(11): 40-44.
[14]	汤健, 孙春来, 毛克峰, 贾美英. 基于主元分析和互信息维数约简策略的网络入侵异常检测[J]. 信息网络安全, 2015, 15(9): 78-83.
[15]	李凌云, 敖吉, 乔治, 李剑. 基于微博的安全事件实时监测框架研究[J]. 信息网络安全, 2015, 15(1): 16-23.

基于实例的迁移时间序列异常检测算法研究

Research on Transfer Time Series Anomaly Detection Algorithm Based on Instance

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 4

参考文献 14

相关文章 15

编辑推荐

Metrics

本文评价