基于上下文聚类的云虚拟机异常检测与识别策略

doi:10.3969/j.issn.1671-1122.2018.12.008

摘要/Abstract

摘要：

根据虚拟机的运行特点,文章提出一种基于上下文聚类的虚拟机异常检测策略。该策略采用一种新的聚类初始中心点选取策略将具有相似上下文运行环境的虚拟机实例进行聚集,对影响空间的局部异常因子算法进行增量式改进,针对每一个上下文类簇构建了上下文异常检测模型。对实时采集的虚拟机实例按照其包含的上下文信息将其匹配到相应上下文异常检测模型中,相应的上下文异常检测模型能够对新采集的虚拟机实例进行增量式异常检测。多个数值实验证明文章提出的异常检测模型和识别算法是有效且高效的。

关键词: 云计算, 虚拟机, 异常检测, 上下文聚类, 增量式

Abstract:

According to the characteristics of virtual machine, an exception detection strategy based on context clustering is proposed, which uses a new clustering initial center selection strategy to aggregate virtual machine instances with similar context running environment. Then, the local anomaly factor algorithm that affects space is improved incrementally, and a context anomaly detection model is constructed for each contextual cluster. The real-time acquisition virtual machine is matched to the corresponding context anomaly detection model according to the context information contained. The corresponding context anomaly detection model can incrementally detect the newly collected virtual machine instance. Several numerical experiments show that the proposed anomaly detection model and recognition algorithm are effective and efficient.

Key words: cloud computing, virtual machine, anomaly detection, context clustering, incrementally

中图分类号:

TP309

何利, 姚元辉. 基于上下文聚类的云虚拟机异常检测与识别策略[J]. 信息网络安全, 2018, 18(12): 54-65.

Li HE, Yuanhui YAO. Detection and Recognition Strategy for Anomaly of Cloud Virtual Machine Based on Context Clustering[J]. Netinfo Security, 2018, 18(12): 54-65.

图/表 12

图1

图2

图3

图4

表1

表2

表3

表4

表5

图5

表6

图6

参考文献 26

[1]	PANNU H S, LIU Jianguo, FU Song.AAD: Adaptive Anomaly Detection System for Cloud Computing Infrastructures[C]// IEEE. 31st Symposium on Reliable Distributed Systems, October 8-11, 2012, Irvine, CA, USA. New Jersey: IEEE, 2013: 396-397.
[2]	PANNU H S, LIU Jianguo, GUAN Qiang, et al.AFD: Adaptive Failure Detection System for Cloud Computing Infrastructures[C]// IEEE. 31st International Performance Computing and Communications Conference, December, 1-3, 2012, Austin, TX, USA. New Jersey: IEEE, 2013: 71-80.
[3]	ZHENG Dequan, LI Fenghuan, ZHAO Tiejun. Self-adaptive Statistical Process Control for Anomaly Detection in Time Series[EB/OL]. , 2018-4-11.
[4]	YE Xiaoming, CHEN Xinghua, WANG Haizhou, et al.An Anomalous Behavior Detection Model in Cloud Computing[J]. Tsinghua Science & Technology, 2016, 21(3): 322-332.
[5]	SHIRAZI S N, SIMPSON S, GOUGLIDIS A, et al.Anomaly Detection in the Cloud Using Data Density[C]// IEEE. 9th Internation Conference on Cloud Computing, June 27-July 2, 2016, San Francisco, CA, USA. New Jersey: IEEE, 2016: 616-623.
[6]	GURUMDIMMA N, JHUMKA A, LIAKATA M, et al.CRUDE: Combining Resource Usage Data and Error Logs for Accurate Error Detection in Large-Scale Distributed Systems[C]// IEEE. 35th Symposium on Reliable Distributed Systems, September 26-29, 2016, Budapest, Hungary. Jersey: IEEE, 2016: 51-60.
[7]	SMRITHY G S, MUNIRATHINAM S, BALAKRISHNAN R.Online Anomaly Detection Using Non-parametric Technique for Big Data Streams in Cloud Collaborative Environment[C]// IEEE. 2016 IEEE International Conference on Big Data, December 5-8, 2016, Washington, DC, USA. Jersey: IEEE, 2016: 1950-1955.
[8]	GUAN Qiang, FU Song, DEBARDELEBEN N, et al.Exploring Time and Frequency Domains for Accurate and Automated Anomaly Detection in Cloud Computing Systems[C]// IEEE. 19th Pacific Rim International Symposium on Dependable Computing, December 2-4, 2013, Vancouver, BC, Canada. Jersey: IEEE, 2013: 196-205.
[9]	XIA Yuming, HU Shaoyong, ZHU Shaomin, et al.Research on the Method of Network Attack Detection Based on Convolution Neural Network[J]. Netinfo Security, 2017, 17(11): 32-36.
	夏玉明,胡绍勇,朱少民,等. 基于卷积神经网络的网络攻击检测方法研究[J]. 信息网络安全,2017,17(11):32-36.
[10]	HE Mingliang, CHEN Zemao, ZUO Jin.Cluster Anomaly Detection Algorithm Based on Multi-windows Mechanism[J]. Netinfo Security, 2016, 16(11): 33-39.
	何明亮,陈泽茂,左进. 基于多窗口机制的聚类异常检测算法[J]. 信息网络安全,2016,16(11):33-39.
[11]	PANDEESWARI N, KUMAR G.Anomaly Detection System in Cloud Environment Using Fuzzy Clustering Based ANN[J]. Mobile Networks and Applications, 2016, 21(3): 494-505.
[12]	PANNU H S, LIU Jianguo, FU Song.A Self-Evolving Anomaly Detection Framework for Developing Highly Dependable Utility Clouds[C]// IEEE. 2012 IEEE Global Communications Conference, December 3-7, 2012, Anaheim, CA, USA. Jersey: IEEE, 2012: 1605-1610.
[13]	BREUNIG M M, KRIEGEL H P, NG R T.LOF: Identifying Density-based Local Outliers[C]// ACM. 2000 ACM SIGMOD International Conference on Management of Data, May 15-18, 2000, Dallas, Texas, USA. New York: ACM, 2000: 93-104.
[14]	LIN Weiming.Research on Methods for Detecting Virtual Machine Abnormal Behaviors in Cloud Computing Platforms[D]. Chongqing: Chongqing University, 2014.
	林铭炜. 面向云平台的虚拟机异常行为检测方法研究[D]. 重庆:重庆大学,2014.
[15]	ZHAO Gang, YAO Xingren.Anomaly Detection Model Based on User Portrait[J]. Netinfo Security, 2017, 17(7): 18-24.
	赵刚,姚兴仁. 基于用户画像的异常行为检测模型[J]. 信息网络安全,2017,17(7):18-24.
[16]	WANG Jun, XIA Limin.Abnormal Behavior Detection in Videos Using Deep Learning[J]. Cluster Computing, 2018(11): 1-11.
[17]	GNOUMA M, EJBALI R, ZAIED M.Abnormal Events’ Detection in Crowded Scenes[J]. Multimedia Tools & Applications, 2018(3):1-22.
[18]	SHARIFZADEH S, BIRO I, LOHSE N, et al. Abnormality Detection Strategies for Surface Inspection Using Robot Mounted Laser Scanners[EB/OL]. , 2018-4-12.
[19]	KARAMI A. An Anomaly-based Intrusion Detection System in Presence of Benign Outliers with Visualization Capabilities[EB/OL]. , 2018-4-12.
[20]	FARSHCHI M, SCHNEIDER J G, WEBER I, et al. Metric Selection and Anomaly Detection for cloud Operations Using Log and Metric Correlation Analysis[EB/OL]. , 2018-4-12.
[21]	WANG Tao, XU Jiwei, ZHANG Wenbo, et al. Self-adaptive Cloud Monitoring with Online Anomaly Detection[EB/OL]. , 2018-4-12.
[22]	ZHOU Yizhou, WANG Bin, XIE Xiaoquan.Design of Software Defined Intrusion Detection System in Cloud[J]. Netinfo Security, 2015, 15(9): 191-195.
	周益周,王斌,谢小权. 云环境下软件定义入侵检测系统设计[J]. 信息网络安全,2015,15(9):191-195.
[23]	JIN Wen, TUNG A K H, HAN Jiawei, et al. Ranking Outliers Using Symmetric Neighborhood Relationship[C]//Springer. Pacific-Asia Conference on Knowledge Discovery and Data Mining, April 9-12, 2006, Singapore. Heidelberg: Springer, 3918: 577-593.
[24]	WHITE T, CUTTING D.Hadoop: the Definitive Guide[J]. Sebastopol: O’reilly Media Inc, 2009.
[25]	KARAU H, KONWINSKI A, WENDELL P, et al.Learning Spark: Lightning-Fast Big Data Analysis[M]. Sebastopol: O’reilly Media Inc, 201.
[26]	CAMPELLO R J G B. A Fuzzy Extension of the Rand Index and other Related Indexes for Clustering and Classification Assessment[J]. Pattern Recognition Letters, 2007, 28(7): 833-841.

聚类结果	不同种类鸢尾花个数/个
聚类结果	Serosa	Versicolour	Virginica
类簇1	9	0	35
类簇2	0	50	0
类簇3	41	0	15

聚类结果	不同种类鸢尾花个数/个
聚类结果	Serosa	Versicolour	Virginica
类簇1	12	0	47
类簇2	0	50	0
类簇3	38	0	3

聚类结果	不同种类鸢尾花个数
聚类结果	Serosa	Versicolour	Virginica
类簇1	7	0	44
类簇2	0	50	0
类簇3	43	0	6

属性类型	属性名称	属性解释
上下文参数	stageType	运行任务所属阶段类型,如shuffleStage
	ShuffleRead	运行任务的shuffle数据读取量
	shuffleWrite	运行任务的shuffle数据发送量
	remoteSent	运行任务的远程数据发送量
	remoteReceive	运行任务的远程数据接收量
性能指标	User%	虚拟机在用户模型下的CPU使用率
	Mem%	虚拟机的内存使用率
	Ioread/s	虚拟机每秒钟对磁盘读取的数据量
	Iowrite/s	虚拟机每秒钟对磁盘写入的数据量
	Rxbyt/s	虚拟机每秒钟通过网络接收的数据量
	Txbyt/s	虚拟机每秒钟通过网络发送的数据量

在线异常检测	异常检测结果				精确度/%	召回率/%	F值/%
在线异常检测	TP	FP	FN	TN	精确度/%	召回率/%	F值/%
有上下文聚类	26	5	4	55	83.87	86.67	85.25
无上下文聚类	17	11	13	49	60.71	56.67	58.62