Education Security Data Management Using Hierarchical Data Quality Assessment Method

doi:10.3969/j.issn.1671-1122.2019.12.010

Abstract

Abstract:

Multi-source network security data from different platform bring challenges for network supervisors. This paper presents a comprehensive quality evaluation method of big data based on analytic hierarchy process (AHP), establishes a hierarchical data quality assessment model for multi-source system platform, designs a hierarchical data quality metric system, and introduces two new data quality dimensions, richness and exchangeability. This paper analyzed the security data set of a city's comprehensive big data platform for education, and proposed management solutions to improve the data quality.

Key words: security data, hierarchical method, quality assessment

CLC Number:

TP309

Mingzheng WANG, Weiqiang MENG, Jianhua HUANG, Feixia WU. Education Security Data Management Using Hierarchical Data Quality Assessment Method[J]. Netinfo Security, 2019, 19(12): 79-87.

Figures/Tables 6

References 23

[1]	KAHN B K, STRONG D M.Product and Service Performance Model for Information Quality: An Update[C]// MIT. Third Conference on Information Quality(IQ 1998), October 23-25, 1998, Cambridge, MA, USA. Trier: DBLP, 1998: 102-l15.
[2]	CAPPIELLO C, FRANCALANCI C, PERNICI B.Data Quality Assessment from the User’s Perspective[C]// ACM. Proceedings of the 2004 International Workshop on Information Quality in Information Systems. ACM, June 18-20, 2004, Paris, France. New York: ACM, 2004: 68-73.
[3]	ISO/IEC 25012:2008, Software Engineering-Software Product Quality Requirements and Evaluation(SQuaRE)-Data Quality Model[EB/OL]. , 2019-9-15
[4]	ISO/IEC 25024:2015, Systems and Software Engineering-Systems and Software Quality Requirements and Evaluation(SQuaRE)-Measurement of Data Quality[EB/OL]. , 2019-9-15.
[5]	LIANG Wen, LIU Fuxin, CUI Mengxiao, et al.Research and Application of Data Quality Evaluation Model Based on Data Assets[J]. Computer Knowledge and Technology, 2016, 12(30): 241-242, 245.
	梁文,刘夫新,崔梦枭,等.基于数据资产的数据质量评估模型研究及应用[J].电脑知识与技术,2016,12(30):241-242,245.
[6]	WU Lili, ZHANG Bo.Research on the Method of Improving Data Quality in University Data Governance[J]. Journal of Chongqing University of Technology(Natural Science), 2019, 33(8): 150-156.
	巫莉莉,张波.高校数据治理中提升数据质量的方法研究[J].重庆理工大学学报(自然科学), 2019,33(8):150-156.
[7]	DAI Chaofan, LIU Lihua, ZENG Saihong, et al.On Military Data Quality Management[J]. Journal of Command and Control, 2016, 2(4): 322-328.
	戴超凡,刘丽华,曾赛红,等.军事数据质量管理研究[J].指挥与控制学报,2016,2(4):322-328.
[8]	REIMER A P, MILINOVICH A, MADIGAN E A.Data Quality Assessment Framework to Assess Electronic Medical Record Data for Use in Research[J]. International Journal of Medical Informatics, 2016, 90(6): 40-47.
[9]	PUTTKAMMER N, BASEMAN J G, DEVINE E B, et al.An Assessment of Data Quality in a Multi-site Electronic Medical Record System in Haiti[J]. International Journal of Medical Informatics, 2015, 86(1): 104-116.
[10]	WEISKOPF N G, HRIPCSAK G, SWAMINATHAN S, et al.Defining and Measuring Completeness of Electronic Health Records for Secondary Use[J]. Journal of Biomedical Informatics, 2013, 46(5): 830-836.
[11]	IMMONEN A, PÄÄKKÖNEN P, OVASKA E. Evaluating the Quality of Social Media Data in Big Data Architecture[J]. IEEE Access, 2015, 3(10): 2028-2043.
[12]	YANG Kang.Design and Implementation of Fund Data Quality Management Platform Based on Evaluation Model[D]. Hangzhou: Zhejiang University.
	杨康. 基于评价模型的基金数据质量管理平台的设计和实现[D].杭州,浙江大学,2015.
[13]	ARBESSER C, SPECHTENHAUSER F, MUHLBACHER T, et al.Visplause: Visual Data Quality Assessment of Many Time Series Using Plausibility Checks[J]. IEEE Transactions on Visualization & Computer Graphics, 2017, 23(1): 641-650.
[14]	SERHANI M A, KASSABI H T, TALEB I, et al.An Hybrid Approach to Quality Evaluation Across Big Data Value Chain[C]//IEEE. International Congress on Big Data(BigData Congress), June 27- July 2, 2016, San Francisco, CA, USA. New York: IEEE, 2016: 418-425.
[15]	ABARZA R G, MOTZ R, URRUTIA A.Quality Assessment Using Data Ontologies[C]// IEEE. International Conference of the Chilean Computer Science Society, November 8-14, 2014, Talca, Maule, Chile. New York: IEEE Computer Society, 2014: 30-33.
[16]	HUANG Hengjun, TAO Ran, FU Deyin.Business Register Database Revision: Internet Data Sources and Data Quality Assessment[J]. Statistical Research, 2017, 34(1): 12-22.
	黄恒君,陶然,傅德印.单位名录库更新:互联网大数据源及其数据质量评估[J].统计研究,2017,34(1):12-22.
[17]	ZHAO Xing, LI Shijun, YU Wei, et al.Research on Web Data Source Quality Assessment Method in Big Data[J]. Computer Engineering, 2017, 34(2): 48-56.
	赵星,李石君,余伟,等.大数据环境下 Web 数据源质量评估方法研究[J].计算机工程,2017,34(2):48-56.
[18]	LI Feifei, ZHANG Jianhua, ZHU Mengshuai, et al. Discussion on Agricultural Data Quality and Evaluation Methods, 2017, 45(36): 221-223, 258.
	李斐斐,张建华,朱孟帅,等.农业数据质量及评估方法探讨[J].安徽农业科学,2017,45:(36)221-223,258.
[19]	LIU Fang, LI Min, REN Hongmin, et al.Data Quality Evaluation Method Based on Rule Base[J]. Computer Systems & Applications, 2017, 26(11): 165-169.
	刘芳,李敏,任洪敏,等.基于规则库的数据质量评估方法[J].计算机系统应用,2017,26(11):165-169.
[20]	MA Qian, GU Yu, ZHANG Tiancheng, et al.A Heterogeneous Multi-source Multi-mode Sensory Data Acquisition Method Based on Data Quality[J]. Chinese Journal of Computers, 2013, 36(10): 2120-2131.
	马茜,谷峪,张天成,等.一种基于数据质量的异构多源多模态感知数据获取方法[J].计算机学报,2013,36(10):2120-2131.
[21]	YANG Ru.Determining the Weights of Performance Evaluation Indexes for College Teachers Based on AHP Method[J]. Modern Economic Information, 2010, (12): 63-63, 65.
	杨茹. 基于AHP方法的高校教师绩效考核指标权重的确定[J].现代经济信息,2010,(12):63-63,65.
[22]	QI Yuan, WANG Feihan, GAO Hongmei, et al.Research on Data Quality Assessment for Big Data in Public Safety Based on Analytic Hierarchy Process[J]. Modern Information Technology, 2019, 3(3): 139-144.
	漆源,王非函,高洪美,等.基于层次分析法的公共安全大数据质量评估研究[J].现代信息科技,2019,3(3):139-144.
[23]	YAN Hao, QIU Hangping, DIAO Xingchun, et al.Comprehensive Data Quality Assessment Based on Improved Analytic Hierarchy Process[J]. Journal of Computer Applications, 2014, 34(A01): 287-290.
	严浩,裘杭萍,刁兴春,等.基于改进层次分析的数据质量综合评估[J].计算机应用,2014,34(A01):287-290.

维度	子维度	描述	子维度计算参数说明
完整性	属性完整性	属性数据值的完整程度	A=数据值不空的记录个数 B=被度量的记录个数
一致性	记录一致性	关联属性一致的比例	A=一致的记录个数 B=关联属性均不为空或null值的记录个数
精确性	属性精确性	属性数据值的精确程度	A=属性中满足精度需求的数据值数 B=属性中不为空或者null值的数据值数
规范性	属性规范性	属性数据值的规范程度	A=属性中满足规范需求的数据值数 B=属性中数据值数
时效性	采集时效性	采集周期	A=某时间段实际采集次数 B=某时间段期望的采集次数
时效性	采集时效性	最近采集时间	A=期望的最近采集时间与当前时间的间隔 B=实际最近采集时间与当前时间的间隔
丰富性	记录丰富性	记录的丰富程度	A=数据集中记录个数 B=数据集中期望的数据个数
丰富性	属性丰富性	属性列的丰富程度	A=数据集中属性列个数 B=数据集中期望的属性列个数
可恢复性	记录正确恢复程度	记录可被正确恢复的程度	A=数据集中成功被恢复的记录数 B=数据集中需要被恢复的记录数
可交换性	必需字段比例	具有数据交换属性的程度	A=数据集中具有的数据交换属性的个数 B=数据交换需要的属性个数
可交换性	属性类型符合度	满足数据交换规则的程度	A=属性类型符合数据交换要求的个数 B=数据集中具有数据交换属性的个数

数据源	数据集	数据集质量得分	数据源质量得分
数据平台1	路由器日志记录	85.36	85.36
数据平台2	路由器日志记录	93.92	96.64
	防火墙日志记录	97.04
	出口流量监测数据	98.95
数据平台3	路由器日志记录	93.04	90.25
	防火墙日志记录	95.57
	WAF日志记录	96.97
	交换机日志记录	87.70
	VPN日志记录	94.72
	负载均衡日志记录	91.17
	出口流量监测数据	69.48
	内网流量监测数据	93.37
数据平台4	路由器日志记录	98.80	95.97
	防火墙日志记录	99.97
	交换机日志记录	93.61
	出口流量监测数据	96.20
	内网流量监测数据	91.09
数据平台5	防火墙日志记录	92.59	92.59

维度	子维度	权重
完整性	属性完整性	100%
一致性	属性一致性	100%
精确性	属性精确性	100%
规范性	属性规范性	100%
时效性	采集周期	50%
时效性	最近采集时间	50%
丰富性	记录丰富性	40%
丰富性	属性丰富性	60%
可恢复性	记录正确恢复程度	100%
可交换性	必需属性比例	50%
可交换性	属性类型符合度	50%