一种基于多模型融合的隐蔽隧道和加密恶意流量检测方法

doi:10.3969/j.issn.1671-1122.2024.05.004

摘要/Abstract

摘要：

高级持续威胁APT攻击为了躲避检测，攻击者往往采用加密恶意流量和隐蔽隧道等策略隐匿恶意行为，从而增加检测的难度。目前大多数检测DNS隐蔽隧道的方法基于统计、频率、数据包等特征，这种方法不能很好地进行实时检测，从而导致数据泄露，因此，需要根据单个DNS请求进行检测而不是对流量进行统计后再检测，才能够实现实时且可靠的检测，当系统判定单个DNS请求为隧道流量，便可做出响应，进而避免数据泄露。而现有的加密恶意检测方法存在无法完整提取流量特征信息、提取特征手段单一、特征利用少等问题。因此，文章提出了基于多模型融合的隐蔽隧道加密恶意流量检测方法。对于DNS隐蔽隧道，文章提出了MLP、1D-CNN、RNN模型融合的检测方法并根据提出的数学模型计算融合结果，该方法能够对隐蔽隧道实时监测，进一步提高检测的整体准确率。对于加密恶意流量，文章提出了1D-CNN、LSTM模型的并行融合的检测方法，并行融合模型能够更加全面地提取特征信息，反应流量数据的全貌，进而提高模型的检测精度。

关键词: 加密恶意流量检测, DNS隐蔽隧道检测, 多模型融合

Abstract:

To evade detection, advanced persistent threat(APT) attackers often employ strategies such as encrypted malicious traffic and covert tunnels to conceal malicious activities, thereby increasing the difficulty of detection. Currently, most methods for detecting DNS covert tunnels are based on characteristics such as statistics, frequency, and packets. These methods are not well-suited for real-time detection, which can lead to data leaks. Therefore, it is necessary to detect based on individual DNS requests rather than performing statistical analysis on traffic, to achieve real-time and reliable detection. When the system determines that a single DNS request is tunnel traffic, it can respond accordingly to prevent data leaks. However, existing methods for detecting encrypted malicious traffic have issues such as the inability to fully extract traffic feature information, limited means of feature extraction, and underutilization of features. Thus, this paper proposed a method for detecting covert tunnel malicious encrypted traffic based on multi-model fusion. For DNS covert tunnels, the paper proposed a detection method that fused MLP, 1D-CNN, and RNN models and calculates the fusion results based on a proposed mathematical model. This method can monitor covert tunnels in real-time, further improving the overall detection accuracy. For encrypted malicious traffic, the paper proposed a parallel fusion detection method combining 1D-CNN and LSTM models. The parallel fusion model can more comprehensively extract feature information and reflect the full scope of the traffic data, thereby enhancing the detection accuracy of the model.

Key words: encrypt malicious traffic detection, DNS hidden tunnel detection, multi model fusion

中图分类号:

TP309

顾国民, 陈文浩, 黄伟达. 一种基于多模型融合的隐蔽隧道和加密恶意流量检测方法[J]. 信息网络安全, 2024, 24(5): 694-708.

GU Guomin, CHEN Wenhao, HUANG Weida. A Covert Tunnel and Encrypted Malicious Traffic Detection Method Based on Multi-Model Fusion[J]. Netinfo Security, 2024, 24(5): 694-708.

图/表 28

图1

图2

图3

图4

图5

图6

表1

表2

表3

图7

表4

图8

表5

图9

表6

图10

表7

表8

图11

表9

图12

表10

图13

表11

图14

表12

图15

表13

参考文献 37

[1]	REZAEI S, LIU XIN. Deep Learning for Encrypted Traffic Classification: An Overview[J]. IEEE Communications Magazine, 2019, 57(5): 76-81. doi: 10.1109/MCOM.2019.1800819
[2]	TANG Zhengzhi, ZENG Xuewen, GUO Zhichuan, et al. Malware Traffic Classification Based on Recurrence Quantification Analysis[J]. International Journal of Networking and Security, 2020, 22(3): 449-459.
[3]	WU Kemeng, ZHANG Yongzheng, TAO Yin. TDAE: Autoencoder-Based Automatic Feature Learning Method for the Detection of DNS Tunnel[C]// IEEE. 2020 IEEE International Conference on Communications (ICC)2020. New York: IEEE, 2020: 1-7.
[4]	YAN Chuyu, GAO Songfeng, WANG Baohui. Research on Encrypted Malicious Traffic Detection[J]. New Industrialization, 2021, 11(10): 59-61.
	闫楚玉, 高嵩峰, 王宝会. 加密恶意流量检测研究[J]. 新型工业化, 2021, 11(10):59-61.
[5]	ANDERSON B, MCGREW D. Identifying Encrypted Malware Traffic with Contextual Flow Data[C]// ACM. Proceedings of the 2016 ACM Workshop on Artificial Intelligence and Security. New York: ACM, 2016: 35-46.
[6]	LI Yanmiao, GUO Hao, HOU Jiangang, et al. A Survey of Encrypted Malicious Traffic Detection[C]// IEEE. 2021 International Conference on Communications, Computing, Cybersecurity, and Informatics (CCCI). New York: IEEE, 2021: 1-7.
[7]	WANG Wei, ZHU Ming, ZENG Xuewen, et al. Malware Traffic Classification Using Convolutional Neural Network for Representation Learning[C]// IEEE. 2017 International Conference on Information Networking (ICOIN). New York: IEEE, 2017: 712-717.
[8]	RADIVILOVA T, KIRICHENKO L, AGEYEV D, et al. Decrypting SSL/TLS Traffic for Hidden Threats Detection[C]// IEEE. 2018 IEEE 9th International Conference on Dependable Systems, Services and Technologies (DESSERT). New York: IEEE, 2018: 143-146.
[9]	KILIC F, ECKERT C. iDeFEND: Intrusion Detection Framework for Encrypted Network Data[C]// Springer. Cryptology and Network Security. Heidelberg: Springer, 2015: 111-118.
[10]	ZHANG Han, PAPADOPOULOS C, MASSEY D. Detecting Encrypted Botnet Traffic[C]// IEEE. 2013 Proceedings IEEE INFOCOM. New York: IEEE, 2013: 3453-3458.
[11]	HU Bin. Research on Detection of Malicious SSL/TLS Encrypted Traffic[D]. Shanghai: Shanghai Jiao Tong University, 2020.
	胡斌. 恶意 SSL/TLS 加密流量检测研究[D]. 上海: 上海交通大学, 2020.
[12]	SHEKHAWAT A S. Analysis of Encrypted Malicious Traffic[EB/OL]. (2018-05-18)[2023-09-05]. https://scholarworks.sjsu.edu/etd_projects/622/.
[13]	LEE I, ROH H, LEE W. Encrypted Malware Traffic Detection Using Incremental Learning[C]// IEEE. IEEE INFOCOM 2020-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS). New York: IEEE, 2020: 1348-1349.
[14]	LI Li, REN Yifei, LOU Jiapeng. A Method for Malicious Encrypted Traffic Detection Based on Ensemble Learning[J]. Journal of Beijing Electronic Science and Technology Institute, 2021, 29(2): 8-16.
	李莉, 任逸飞, 娄嘉鹏. 一种基于集成学习的恶意加密流量检测方法[J]. 北京电子科技学院学报, 2021, 29(2):8-16.
[15]	LUO Ziming, XU Shubin, LIU Xiaodong. A TLS Malicious Encrypted Traffic Detection Scheme Based on Machine Learning[J]. Journal of Network and Information Security, 2020, 6(1): 77-83.
	骆子铭, 许书彬, 刘晓东. 基于机器学习的TLS恶意加密流量检测方案[J]. 网络与信息安全学报, 2020, 6(1):77-83.
[16]	JIANG Peng. Analysis and Detection of C&C Encrypted Channel Network Behavior Under APT Attacks[D]. Shanghai: Shanghai Normal University, 2018.
	姜鹏. APT 攻击下的 C&C 加密信道网络行为分析与检测[D]. 上海: 上海师范大学, 2018.
[17]	PRASSE P, MACHLICA L, PEVNÝ T, et al. Malware Detection by Analysing Network Traffic with Neural Networks[C]// IEEE. 2017 IEEE Security and Privacy Workshops (SPW). New York: IEEE, 2017: 205-210.
[18]	CHEN Lin, JIANG Yixi, KUANG Xiaoyun, et al. Deep Learning Detection Method of Encrypted Malicious Traffic for Power Grid[C]// IEEE. 2020 IEEE International Conference on Energy Internet (ICEI). New York: IEEE, 2020: 86-91.
[19]	ZENG Yi, GU Huaxi, WEI Wenting, et al. $ Deep-Full-Range $: A Deep Learning Based Network Encrypted Traffic Classification and Intrusion Detection Framework[J]. IEEE Access, 2019, 7: 45182-45190.
[20]	WANG Yue, ZHOU Anmin, LIAO Shan, et al. A Comprehensive Survey on DNS Tunnel Detection[EB/OL]. (2021-10-09)[2023-09-07]. https://www.sciencedirect.com/science/article/pii/S1389128621003248.
[21]	CROTTI M, DUSI M, GRINGOLI F, et al. Detecting Http Tunnels with Statistical Mechanisms[C]// IEEE. 2007 IEEE International Conference on Communications. New York: IEEE, 2007: 6162-6168.
[22]	DUSI M, CROTTI M, GRINGOLI F, et al. Tunnel Hunter: Detecting Application-Layer Tunnels with Statistical Fingerprinting[J]. Computer Networks, 2009, 53(1): 81-97.
[23]	CASAS P, MAZEL J, OWEZARSKI P. MINETRAC: Mining Flows for Unsupervised Analysis & Semi-Supervised Classification[C]// IEEE. 2011 23rd International Teletraffic Congress (ITC). New York: IEEE, 2011: 87-94.
[24]	WANG Hao. Research on Anomaly DNS Traffic Detection Based on Machine Learning[D]. Nanjing: Nanjing University of Posts and Telecommunications, 2019.
	王浩. 基于机器学习的异常 DNS 流量检测研究[D]. 南京: 南京邮电大学, 2019.
[25]	SAKARKAR G, KOLEKAR M K H, PAITHANKAR K, et al. Advance Approach for Detection of DNS Tunneling Attack from Network Packets Using Deep Learning Algorithms[J]. Advances in Distributed Computing and Artificial Intelligence Journal, 2021, 10(3): 241-266.
[26]	SHERIDAN S, KEANE A. Detection of DNS Based Covert Channels[EB/OL]. (2015-01-01)[2023-09-05]. https://www.researchgate.net/publication/282931276_Detection_of_DNS_based_covert_channels.
[27]	LIU Xiaolei, ZHANG Qiongyin, REN Lei, et al. Exploration of DNS Tunneling Trojan Detection Technology Based on Communication Behavior Analysis[J]. Science and Technology Information, 2018, 16(34): 17-18.
	刘晓蕾, 张琼尹, 任磊, 等. 基于通信行为分析的DNS隧道木马检测技术探究[J]. 科技资讯, 2018, 16(34):17-18.
[28]	NADLER A, AMINOV A, SHABTAI A. Detection of Malicious and Low Throughput Data Exfiltration over the DNS Protocol[J]. Computers & Security, 2019, 80: 36-53.
[29]	QI Cheng, CHEN Xiaojun, XU Cui, et al. A Bigram Based Real Time DNS Tunnel Detection Approach[J]. Procedia Computer Science, 2013, 17: 852-860.
[30]	YU Bin, SMITH L, THREEFOOT M, et al. Behavior Analysis Based DNS Tunneling Detection and Classification with Big Data Technologies[C]// IoTBD. Behavior Analysis Based DNS Tunneling Detection and Classification with Big Data Technologies. Rome: IoTBD, 2016: 284-290.
[31]	CHEN Shaojie, LANG Bo, LIU Hongyu, et al. DNS Covert Channel Detection Method Using the LSTM Model[EB/OL]. (2021-01-28)[2023-09-05]. https://www.sciencedirect.com/science/article/pii/S0167404820303680.
[32]	WANG Qi, XIE Kun, MA Yan, et al. DNS Tunnel Detection Based on Log Statistical Features[J]. Journal of Zhejiang University (Engineering Science), 2020, 54(9): 1753-1760.
	王琪, 谢坤, 马严, 等. 基于日志统计特征的 DNS 隧道检测[J]. 浙江大学学报(工学版), 2020, 54(9):1753-1760.
[33]	LIN Huaqing, LIU Gao, YAN Zheng. Detection of Application-Layer Tunnels with Rules and Machine Learning[C]// Springer. International Conference on Security, Privacy and Anonymity in Computation, Communication and Storage. Heidelberg: Springer, 2019: 441-455.
[34]	ZHANG Meng, SUN Haoliang, YANG Peng. DNS Covert Channel Recognition Based on Improved Convolutional Neural Network[J]. Journal on Communications, 2020, 41(1): 169-179. doi: 10.11959/j.issn.1000-436x.2020017
	张猛, 孙昊良, 杨鹏. 基于改进卷积神经网络识别 DNS 隐蔽信道[J]. 通信学报, 2020, 41(1):169-179. doi: 10.11959/j.issn.1000-436x.2020017
[35]	FARNHAM G, ATLASIS A. Detecting DNS Tunneling[J]. SANS Institute InfoSec Reading Room, 2013, 9: 1-32.
[36]	CHEN Yang, LI Xiaoyong. A High Accuracy DNS Tunnel Detection Method without Feature Engineering[C]// IEEE. 2020 16th International Conference on Computational Intelligence and Security (CIS). New York: IEEE, 2020: 374-377.
[37]	GARCIA S, GRILL M, STIBOREK J, et al. An Empirical Comparison of Botnet Detection Methods[J]. Computers & Security, 2014, 45: 100-123.

模型	准确率	召回率	精确率	F1分数
MLP_1	0.966250	0.932510	0.999989	0.965072
MLP_2	0.966350	0.932720	0.999978	0.965179
MLP_3	0.970900	0.941830	0.999968	0.970029
MLP_4	0.978960	0.957960	0.999958	0.978509

模型	准确率	召回率	精确率	F1分数
1D-CNN_1	0.981635	0.963280	0.999990	0.981292
1D-CNN_2	0.981640	0.963370	0.999907	0.981298
1D-CNN_3	0.987630	0.975270	0.999990	0.987475
1D-CNN_4	0.981625	0.963250	1.000000	0.981281

模型	准确率	召回率	精确率	F1分数
RNN_1	0.967180	0.934420	0.999936	0.966068
RNN_2	0.966165	0.932390	0.999936	0.964982
RNN_3	0.981650	0.963400	0.999990	0.981309
RNN_4	0.980765	0.961600	0.999927	0.980389

模型	准确率	召回率	精确率	F1分数
MLP	0.978960	0.957960	0.999958	0.978509
1D-CNN	0.987630	0.975270	0.999990	0.987475
RNN	0.981650	0.963400	0.999896	0.981309
融合模型	0.990975	0.982560	0.999380	0.990898

模型	准确率	召回率	精确率	F1分数
1D-CNN	0.981596	0.984155	0.975213	0.979664
LSTM	0.980462	0.983999	0.972932	0.978434
串行融合模型	0.985580	0.988872	0.979297	0.984061
并行融合模型	0.990101	0.991140	0.987046	0.989089