Cluster Anomaly Detection Algorithm Based on Multi-windows Mechanism

doi:10.3969/j.issn.1671-1122.2016.11.006

Abstract

Abstract:

This paper analyses the weaknesses of cluster anomaly detection algorithm based on single-window, takes advantage of weigh value, similarity, local density and other concepts to conduct affiliation search and abnormal merging on potential abnormal point obtained by single-window algorithm. Moreover, a dataflow anomaly detection algorithm based on multi-window mechanism is designed. This algorithm firstly conducts primary cluster detection to preprocessed dataflow with improved K-means cluster algorithm in single window and then conduct second judge to the results. For the potential abnormal point detected by single-window algorithm, similarity principle is adopted to conduct normal cluster affiliation search to exclude misjudges, other conceptions like local density is adopted to conduct abnormal merging to the rest potential abnormal points to exclude normal points again. Lastly, the time weigh value is used to obtain final abnormal data comprehensively from the detection results of several dataflow windows. The simulation shows that this algorithm has advantage over single-window cluster anomaly detection algorithm on detection rate and misjudge rate.

Key words: single window, multi-windows, data flow, anomaly detection

CLC Number:

TP309

Mingliang HE, Zemao CHEN, Jin ZUO. Cluster Anomaly Detection Algorithm Based on Multi-windows Mechanism[J]. Netinfo Security, 2016, 16(11): 33-39.

Figures/Tables 12

References 19

[1]	SHIE B E, YU P S, TSENG V S.Efficient Algorithms for Mining Maximal High Utility Itemsets from Data Streams with Different Models[J]. Expert Systems with Applications, 2012, 39(17): 12947-12960.
[2]	吴晓平,周舟,李洪成. Spark框架下基于无指导学习环境的网络流量异常检测研究与实现[J]. 信息网络安全,2016(6):1-7.
[3]	Angiulli F, Fassetti F.Detecting Distance-based Outliers in Streams of Data[C]//ACM. The 16th ACM Conference on Information and Knowledge Management, November 6-10, 2007, Lisbon, Portugal. New York: ACM, 2007: 811-820.
[4]	姚晨. 高纬数据流的异常检测[D]. 成都:电子科技大学,2011.
[5]	陈晓,赵晶玲. 大数据处理中混合型聚类算法的研究与实现[J]. 信息网络安全,2015(4):45-49.
[6]	程军锋,王治和,刘佳,等. 一种基于滑动窗口的一趟数据流聚类算法[J]. 首都师范大学学报,2014,35(4):38-40.
[7]	朱琳,刘晓东,朱参世. 基于衰减滑动窗口数据流聚类算法研究[J]. 计算机工程与设计,2012,33(7):2659-2796.
[8]	张付霞,蒋朝惠. 一种基于网格聚类的查询隐私匿名算法研究[J]. 信息网络安全,2015(8):53-58.
[9]	田俊锋,王惠然,刘玉玲. 基于属性排序的入侵特征缩减方法研究[J]. 计算机研究与发展,2006,43(S2):565-569.
[10]	BRUHA I.Pre-and Post-Processing in Machine Learning and Data Mining[J]. Machine Learning and Its Applications, 2010, 18(3): 258-266.
[11]	陈才杰. 粗糙集理论在知识发现数据预处理中的研究与应用[D]. 武汉:武汉理工大学,2014.
[12]	MACQUEEN J. Some Methods for Classification and Analysis of Multivariate Observations[EB/OL]. , 2016-6-20.
[13]	TZORTZIS G,LIKAS A.The Minmax k-means Clustering Algorithm[J]. Pattern Recognition,2011, 44(4): 866-876.
[14]	蒋大宇. 快速有效的并行二分K均值算法[D]. 哈尔滨:哈尔滨工程大学,2013.
[15]	朱建宇. K均值算法研究及其应用[D]. 大连:大连理工大学,2013.
[16]	李旬,徐剑,焦英楠,等. 基于异常特征的社交网页检测技术研究[J]. 信息网络安全,2015(5):41-46.
[17]	LI Huafu.MHUI-max: An Efficient Algorithm for Discovering High-utility Itemsets from Data Streams[J]. Journal of Information Science, 2011, 37(5): 532-545.
[18]	程转流,胡为成. 滑动窗口模型下的概率数据流聚类[J]. 计算机工程与应用,2011,47(4):141-145.
[19]	AHMED C F, TANBEER S K, JEONG B S, et al.Interactive Mining of High Utility Patterns over Data Streams[J]. Expert Systems with Application, 2012, 39(15): 11979-11991.