IoT Anomaly Detection Model Based on Cost-Sensitive Learning

doi:10.3969/j.issn.1671-1122.2023.11.010

Abstract

Abstract:

Aiming at the problem of data imbalance in current abnormal detection algorithms for Internet of Things (IoT), which leads to incomplete feature learning and subsequently affects the detection performance of minority class attack samples, this article proposed a cost-sensitive abnormal detection model for IoT, called CS-CTIAD. The model used convolutional neural networks and Transformers to comprehensively learn the spatial and temporal features of IoT traffic, alleviating the problem of incomplete feature learning of minority class attack samples by a single model; at the same time, cost sensitive learning was introduced in the model training process, dynamically adjusting the loss weights of minority and majority classes to prevent the classifier from ignoring minority class attack samples due to data imbalance, thus improving the recognition rate of minority class attack samples. The test results on the CSE-CIC-IDS2018 and IoT-23 datasets demonstrate a significant improvement in the detection performance of minority class attack samples. Compared with existing work, the proposed method achieves the best overall evaluation metrics (accuracy, precision, recall, F1).

Key words: internet of things, anomaly detection, deep learning, cost-sensitive, class imbalance

CLC Number:

TP309

LIAO Liyun, ZHANG Bolei, WU Lifa. IoT Anomaly Detection Model Based on Cost-Sensitive Learning[J]. Netinfo Security, 2023, 23(11): 94-103.

Figures/Tables 9

References 23

[1]	JSOF. CVE-2020-11896/CVE-2020-11898 Whitepaper[EB/OL]. (2020-10-25)[2023-05-11]. https://www.jsof-tech.com/wp-content/u-ploads/2020/08/Ripple20_CVE-2020-11901-August20.pdf.
[2]	FU Lidong, ZHANG Wenbo, TAN Xiaobo, et al. An Algorithm for Detection of Traffic Attribute Exceptions Based on Cluster Algorithm in Industrial Internet of Things[J]. IEEE Access, 2021, 9: 53370-53378. doi: 10.1109/ACCESS.2021.3068756 URL
[3]	LIU Xiangyu, LU Tianliang, DU Yanhui, et al. Lightweight IoT Intrusion Detection Method Based on Feature Selection[J]. Netinfo Security, 2023, 23(1): 66-72.
	刘翔宇, 芦天亮, 杜彦辉, 等. 基于特征选择的物联网轻量级入侵检测方法[J]. 信息网络安全, 2023, 23(1):66-72.
[4]	FERRAG A M, MAGLARAS L, MOSCHOYIANNIS S, et al. Deep Learning for Cyber Security Intrusion Detection: Approaches, Datasets, and Comparative Study[EB/OL]. [2023-07-16]. https://doi.org/10.1016/j.jisa.2019.102419.
[5]	YIN Ying, ZHOU Zhihong, YAO Lihong. Research on LSTM-Based CAN Intrusion Detection Model[J]. Netinfo Security, 2022, 22(12): 57-66.
	银鹰, 周志洪, 姚立红. 基于LSTM的CAN入侵检测模型研究[J]. 信息网络安全, 2022, 22(12):57-66.
[6]	TESFAHUN A, BHASKARI D L. Intrusion Detection Using Random Forests Classifier with SMOTE and Feature Reduction[C]// IEEE. 2013 International Conference on Cloud & Ubiquitous Computing & Emerging Technologies. New York: IEEE, 2013: 127-132.
[7]	WU Shuguang, WANG Hongyan, WANG Yu, et al. Application of SOINN Based Undersampling Method in Network Intrusion Detection[J]. Modern Electronics Technique, 2022, 45(21): 88-92.
	吴署光, 王宏艳, 王宇, 等. 基于SOINN的欠采样方法在网络入侵检测中的应用[J]. 现代电子技术, 2022, 45(21):88-92.
[8]	KARATAS G, DEMIR O, SAHINGOZ O K. Increasing the Performance of Machine Learning-Based IDSs on an Imbalanced and Up-to-Date Dataset[J]. IEEE Access, 2020, 8: 32150-32162. doi: 10.1109/Access.6287639 URL
[9]	LIU Lan, WANG Pengcheng, LIN Jun, et al. Intrusion Detection of Imbalanced Network Traffic Based on Machine Learning and Deep Learning[J]. IEEE Access, 2021, 9: 7550-7563. doi: 10.1109/Access.6287639 URL
[10]	HUANG Shuokang, LEI Kai. IGAN-IDS: An Imbalanced Generative Adversarial Network towards Intrusion Detection System in Ad-Hoc Networks[EB/OL]. (2020-08-01)[2023-07-16]. https://doi.org/10.1016/j.adhoc.2020.102177.
[11]	ANDRESINI G, APPICE A, ROSE L D, et al. GAN Augmentation to Deal with Imbalance in Imaging-Based Intrusion Detection[J]. Future Generation Computer Systems, 2021, 123: 108-127. doi: 10.1016/j.future.2021.04.017 URL
[12]	JIANG Kaiyuan, WANG Wenya, WANG Aili, et al. Network Intrusion Detection Combined Hybrid Sampling With Deep Hierarchical Network[J]. IEEE Access, 2020, 8: 32464-32476. doi: 10.1109/Access.6287639 URL
[13]	MA Xiangyu, SHI Wei. AESMOTE: Adversarial Reinforcement Learning with SMOTE for Anomaly Detection[J]. IEEE Transactions on Network Science and Engineering, 2021, 8(2): 943-956. doi: 10.1109/TNSE.2020.3004312 URL
[14]	DING Hongwei, CHEN Leiyang, DONG Liang, et al. Imbalanced Data Classification: A KNN and Generative Adversaryal Networks-Based Hybrid Approach for Intrusion Detection[J]. Future Generation Computer Systems, 2022, 131: 240-254. doi: 10.1016/j.future.2022.01.026 URL
[15]	TELIKANI A, GANDOMI A H. Cost-Sensitive Stacked Auto-Encoders for Intrusion Detection in the Internet of Things[EB/OL]. [2023-07-16]. https://doi.org/10.1016/j.iot.2019.100122.
[16]	FUQUA D, RAZZAGHI T. A Cost-Sensitive Convolution Neural Network Learning for Control Chart Pattern Recognition[EB/OL]. (2020-07-15)[2023-07-16]. https://doi.org/10.1016/j.eswa.2020.113275.
[17]	KIM Y. Convolutional Neural Networks for Sentence Classification[C]// ACL. 2014 Empirical Methods in Natural Language Processing(EMNLP). Stroudsburg: ACL, 2014: 1746-1751.
[18]	LIN Tianyang, WANG Yuxin, LIU Xiangyang, et al. A Survey of Transformers[J]. AI Open, 2022, 3: 111-132. doi: 10.1016/j.aiopen.2022.10.001 URL
[19]	SHARAFALDIN I, LASHKARI A H, GHORBANI A A. Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization[C]// INSTICC. 4th International Conference on Information Systems Security and Privacy (ICISSP). Lisbon: SCITEPRESS, 2018: 108-116.
[20]	GARCIA S, PARMISANO A, ERQUIAGA M J. IoT-23: A Labeled Dataset with Malicious and Benign IoT Network Traffic[EB/OL]. [2023-08-10]. https://zenodo.org/record/4743746.
[21]	SAHU A K, SHARMA S, TANVEER M, et al. Internet of Things Attack Detection Using Hybrid Deep Learning Model[J]. Computer Communications, 2021, 176(3): 146-154. doi: 10.1016/j.comcom.2021.05.024 URL
[22]	MEZINA A, BURGET R, TRAVIESO-GONZÁLEZ C M. Network Anomaly Detection with Temporal Convolutional Network and U-Net Model[J]. IEEE Access, 2021, 9: 143608-143622. doi: 10.1109/ACCESS.2021.3121998 URL
[23]	WU Zihan, ZHANG Hong, WANG Penghai, et al. RTIDS: A Robust Transformer-Based Approach for Intrusion Detection System[J]. IEEE Access, 2022, 10: 64375-64387. doi: 10.1109/ACCESS.2022.3182333 URL

文献	数据集	方法	具体实现
文献[6]	KDD CUP99	特征选择法	信息增益思想
文献[7]	KDD CUP99	数据重采样	SOINN欠采样
文献[8]	CSE-CIC-IDS2018	数据重采样	SMOTE过采样
文献[9]	NSL-KDD, CSE-CIC-IDS2018	数据重采样	DSSTE过采样
文献[10]	NSL-KDD, UNSW-NB15, CICIDS2017	数据重采样	IGAN过采样
文献[11]	KDD CUP99, AAGM17, UNSW-NB15, CICIDS2017	数据重采样	GAN过采样
文献[12]	NSL-KDD, UNSW-NB15	数据重采样	OSS欠采样+SMOTE过采样
文献[13]	NSL-KDD	数据重采样	SMOTE过采样+随机过采样+NearMiss欠采样
文献[14]	KDD CUP99, UNSW-NB15, CICIDS2017	数据重采样	KNN欠采样+TACGAN过采样
文献[15]	KDD CUP99, NSL-KDD	代价敏感学习	代价矩阵+ CSSAE
文献[16]	InsectWingbeatSound	代价敏感学习	代价因子+CNN

流量类别	数量	占比
Benign	200000	23.8298%
DDoS-HOIC	100000	11.9149%
DoS-GoldenEye	30585	3.6442%
SQL Injection	53	0.0063%
DoS-Hulk	93659	11.1594%
Bot	100000	11.9149%
SSH-Bruteforce	94237	11.2282%
Brute Force-XSS	117	0.0139%
DoS-SlowHTTPTest	100000	11.9149%
Brute Force-Web	268	0.0319%
DoS-Slowloris	13475	1.6055%
FTP-BruteForce	100000	11.9149%
DDoS-LOIC-UDP	6682	0.7962%
Infiltration	209	0.0249%

流量类别	数量	占比
Benign	150000	47.2713%
C&C	8582	2.7046%
Okiru	50000	15.7571%
PartOfAHorizontalPortScan	50000	15.7571%
DDoS	50000	15.7571%
HeartBeat	7850	2.4739%
Torii	583	0.1837%
Miral	291	0.0917%
FileDownload	11	0.0035%

真实类别	预测类别
真实类别	攻击	正常
攻击	True Positive（TP）	False Negative（FN）
正常	False Positive（FP）	True Negative（TN）

流量类别	CTIAD		CS-CTIAD
流量类别	精确率	F1	精确率	F1
Benign	99.933 %	99.904 %	99.998 %	99.937 %
DDOS-HOIC	99.993 %	99.997 %	100 %	100 %
DoS-GoldenEye	99.978 %	99.989 %	99.989 %	99.995 %
SQL Injection	75.000 %	75.000 %	78.947 %	85.714 %
DoS-Hulk	99.982 %	99.980 %	99.993 %	99.986 %
Bot	100 %	99.997 %	99.990 %	99.995 %
SSH-Bruteforce	100 %	99.986 %	100 %	99.986 %
Brute Force -XSS	81.818 %	90.000 %	81.818 %	90.000 %
DoS-SlowHTTPTest	99.970 %	99.985 %	99.973 %	99.987 %
Brute Force -Web	76.000 %	84.444 %	77.451 %	86.813 %
DoS-Slowloris	99.950 %	99.975 %	99.950 %	99.975 %
FTP-BruteForce	99.990 %	99.995 %	99.977 %	99.988 %
DDoS-LOIC-UDP	100 %	100 %	100 %	100 %
Infiltration	54.839 %	54.839 %	66.667 %	80.000 %