Research Status and Development Trends on Network Encrypted Traffic Identification

doi:10.3969/j.issn.1671-1122.2019.03.003

Abstract

Abstract:

The rapid growth of network encrypted traffic is changing the threat landscape. How to realize real-time and accurate identification of network encrypted traffic is an important issue in the field of cyberspace security in China. It is also a research hotspot of network behavior analysis, network planning construction and network traffic model. This paper discusses the basic concepts, research progress, evaluation indicators and existing issues of network encrypted traffic identification, and summarizes and forecasts the development trends and challenges of network encrypted traffic identification. This paper can provide reference for further exploration of new methods and technologies in the field of cyberspace security.

Key words: network encrypted traffic, encrypted traffic identification, cyber security, encryption protocol, abnormal encrypted traffic

CLC Number:

TP309

Liangchen CHEN, Shu GAO, Baoxu LIU, Zhigang LU. Research Status and Development Trends on Network Encrypted Traffic Identification[J]. Netinfo Security, 2019, 19(3): 19-25.

Figures/Tables 10

References 45

[1]	ROUGHAN M,SEN S,SPATSCHENC O, et al.Class-of-service Mapping for QoS : A Statistical Signature-based Approach to IP Traffic Classification[C]//ACM. The 4th ACM SIGCOMM Conference on Internet Measurement, October 25 - 27, 2004, Taormina, Sicily, Italy.New York: ACM, 2004: 135-148.
[2]	Cisco. Encrypted Traffic Analytics White Paper[EB/OL]. , 2018-12-31.
[3]	ANDERSON B, MCGREW D.Identifying Encrypted Malware Traffic with Contextual Flow Data[C]//ACM. The 2016 ACM Workshop on Artificial Intelligence and Security, October 28, 2016, Vienna, Austria. New York: ACM, 2016:35-46.
[4]	VELAN P, CERMAK M, CELEDA P, et al.A Survey of Methods for Encrypted Traffic Classification and Analysis[J]. International Journal of Network Management, 2015,25(5):355-374.
[5]	PAN Wubin, CHENG Guang, GUO Xiaojun, et al.Review and Perspective on Encrypted Traffic Identification Research[J]. Journal on Communications, 2016, 37(9):154-167.
	潘吴斌, 程光, 郭晓军,等. 网络加密流量识别研究综述及展望[J]. 通信学报, 2016, 37(9):154-167.
[6]	ZHAO Bo, GUO Hong, LIU Qinrang, et al.Protocol Independent Identification of Encrypted Traffic Based on Weighted Cumulative Sum Test[J]. Journal of Software, 2013,24(6):1334-1345.
	赵博,郭虹,刘勤让,等.基于加权累积和检验的加密流量盲识别算法[J].软件学报,2013,24(6):1334-1345.
[7]	CHENG Guang, CHEN Yuxiang.Identification Method of Encrypted Traffic Based on Support Vector machine[J]. Journal of Southeast University (Natural Science Edition), 2017(4):655-659.
	程光, 陈玉祥. 基于支持向量机的加密流量识别方法[J].东南大学学报(自然科学版), 2017(4):655-659.
[8]	YU Qiang, HUO Hongwei.Algorithms Improving the Storage Efficiency of Deep Packet Inspection[J]. Journal of Software, 2011,22(1):149-163.
	于强,霍红卫.一组提高存储效率的深度包检测算法[J].软件学报,2011,22(1):149-163.
[9]	XU Peng, LIN Sen.Internet Traffic Classification Using C4.5 Decision Tree[J]. Journal of Software, 2009,20(10):2692-2704.
	徐鹏,林森.基于C4.5决策树的流量分类方法[J].软件学报, 2009, 20(10):2692-2704
[10]	SHAHBAZ R, XIN Liu. Deep Learning for Encrypted Traffic Classification: An Overview[EB/OL]. , 2018-12-31.
[11]	WEI Shuning, CHEN Xingru, TANG Yong, et al.Research on the Application of AR-HELM Algorithm in Network Traffic Classification[J].Netinfo Security, 2018,18(1):9-14.
	魏书宁,陈幸如,唐勇,等. AR-HELM算法在网络流量分类中的应用研究[J]. 信息网络安全,2018,18(1):9-14.
[12]	WANG Wei.Research on Identification of Encrypted Network Application Traffic[D]. Zhengzhou: PLA Information Engineering University,2014.
	王炜. 网络应用层加密流量识别技术研究[D]. 郑州:解放军信息工程大学,2014.
[13]	CAO Zigang, XIONG Gang, ZHAO Yong, et al.A Survey on Encrypted Traffic Classification[M]// Springer. Applications and Techniques in Information Security. Heidelberg: Springer Berlin Heidelberg, 2014:73-81.
[14]	LU Gang, GUO Ronghua, ZHOU Ying, et al.Review of Malicious Traffic Feature Extraction[J]. Netinfo Security, 2018,18(9):1-9.
	鲁刚,郭荣华,周颖,等. 恶意流量特征提取综述[J]. 信息网络安全,2018,18(9):1-9.
[15]	MA Ruolong.Research and Implementation of Unknown and Encrypted Traffic Identification Based on Convolutional Neural Network[D].Beijing: Beijing University of Posts and Telecommunications,2018.
	马若龙. 基于卷积神经网络的未知和加密流量识别的研究与实现[D]. 北京:北京邮电大学,2018.
[16]	BONFIGLIO D, MELLIA M, MEO M, et al.Revealing skype Traffic: When Randomness Plays with You[J]. ACM SIGCOMM Computer Communication Review, 2007, 37(4):37-48.
[17]	KORCZYNSKI M, DUDA A.Markov Chain Fingerprinting to Classify Encrypted Traffic[C]//IEEE. IEEE Conference on Computer Communications, April 27- May 2 ,2014, Toronto, ON, Canada.NJ:IEEE, 2014:781-789.
[18]	WANG Wei, CHENG Dongnian.M-serial Test-based Encrypted Traffic Identification[J]. Computer Engineering and Design, 2014, 35(11):3712-3716.
	王炜, 程东年. 基于M-序列检验的加密流量识别[J]. 计算机工程与设计, 2014, 35(11):3712-3716.
[19]	KHAKPOUR A R, LIU A X.An Information-Theoretical Approach to High-Speed Flow Nature Identification[J]. IEEE/ACM Transactions on Networking, 2013, 21(4):1076-1089.
[20]	ZHANG Bo.Research on Technology of Encrypted Traffic Identification Based on Traffic Features[D]. Harbin :Harbin Institute of Technology,2012.
	张波. 基于流特征的加密流量识别技术研究[D].哈尔滨:哈尔滨工业大学, 2012.
[21]	CHEN Wei,HU Lei, YANG Long .Fast Identification Method of Encrypted Traffic Based on Payload Signatures[J]. Computer Engineering, 2012, 38(12):22-25.
	陈伟, 胡磊, 杨龙. 基于载荷特征的加密流量快速识别方法[J]. 计算机工程, 2012, 38(12):22-25.
[22]	CHEN Wei, YU Le, YANG Geng.Detecting Encrypted Botnet Traffic Using Spatial-Temporal Correlation[J].China Communications, 2012(10): 49-59.
	陈伟, 于乐, 杨庚. 使用时空相关性分析检测加密僵尸网络流量[J]. 中国通信, 2012(10): 49-59.
[23]	WANG Pan,CHEN Xuejiao.SAE-based Encrypted Traffic Identification Method[J].Computer Engineering,2018,44( 11) : 140-147,153.
	王攀,陈雪娇. 基于堆栈式自动编码器的加密流量识别方法[J].计算机工程,2018,44(11):140-147,153.
[24]	MENG Juan, YANG Longqi, ZHOU Yuhuan, et al.Encrypted Traffic Identification Based on Sparse Logistical Regression and Extreme Learning Machine[M]// Springer. Proceedings of ELM-2014 . Cham :Springer, Cham, 2015:61-70.
[25]	ZHANG Meng, ZHANG Hongli, ZHANG Bo, et al.Encrypted Traffic Classification Based on an Improved Clustering Algorithm[M] // Springer .Trustworthy Computing and Services. Heidelberg: Springer Berlin Heidelberg, 2013: 124-131.
[26]	CHEN Xuejiao,WANG Pan,YU Jiahui .CNN-based Entrypted Traffic Identification Method[J]. Journal of Nanjing University of Posts and Telecommunications( Natural Science Edition), 2018,38(6):36-41.
	陈雪娇,王攀,俞家辉.基于卷积神经网络的加密流量识别方法[J].南京邮电大学学报(自然科学版),2018,38(6):36-41.
[27]	NGUYEN T T T, ARMITAGE G J. A Survey of Techniques for Internet Traffic Classification Using Machine Learning[J]. IEEE Communications Surveys & Tutorials, 2008, 10(3):56-76.
[28]	ALSHAMMARI R, ZINCIR-HEYWOOD A N. Machine Learning-based Encrypted Traffic Classification: Identifying SSH and Skype[C]//IEEE. 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications, July 8-10, 2009, Ottawa, ON, Canada.NJ:IEEE,2009:1-8.
[29]	ALSHAMMARI R, ZINCIR-HEYWOOD A N . An Investigation on the Identification of VoIP Traffic: Case Study on Gtalk and Skype[C]//IEEE.2010 International Conference on Network & Service Management, October 25-29, 2010, Niagara Falls, ON, Canada.NJ:IEEE, 2010:310-313.
[30]	OKADA Y, ATA S, NAKAMURA N, et al.Comparisons of Machine Learning Algorithms for Application Identification of Encrypted Traffic[C] //IEEE.2011 10th International Conference on Machine Learning & Applications & Workshops, December 18-21, 2011, Honolulu, HI, USA.NJ:IEEE,2011: 358-361.
[31]	ALSHAMMARI R, ZINCIRHEYWOOD A N .Can Encrypted Traffic Be Identified without Port Numbers, IP Addresses and Payload Inspection?[J]. Computer Networks, 2011, 55(6):1326-1350.
[32]	HAFFNER P, SEN S, SPATSCHECK O, et al.ACAS: Automated Construction of Application Signatures[C]//ACM. ACM SIGCOMM Workshop on Mining Network Data, August 26 , 2005, Philadelphia, Pennsylvania, USA. New York: ACM, 2005:197-202.
[33]	KARAGIANNIS T .BLINC: Multilevel Traffic Classification in the Dark[J]. Acm Sigcomm Computer Communication Review, 2005, 35(4):229-240.
[34]	WRIGHT C V, MONROSE F, MASSON G M.Using Visual Motifs to Classify Encrypted Traffic[C]//ACM.The 3rd International Workshop on Visualization for Computer Security, November 3, 2006, Alexandria, Virginia, USA . New York: ACM, 2006: 41-50.
[35]	SCHATZMANN D, MÜhlbauer W,SPYROPOULOS T,et al.Digging into HTTPS: Flow-based Classification of Webmail Traffic[C]//ACM. ACM SIGCOMM Conference on Internet Measurement, November 1 - 30, 2010, Melbourne, Australia . New York: ACM, 2010: 322-327.
[36]	BERMOLEN P, MELLIA M, MEO M, et al.Abacus: Accurate Behavioral Classification of P2P-TV Traffic[J]. Computer Networks, 2011, 55(6):1394-1411.
[37]	XIONG Gang, HUANG Wenting, ZHAO Yong, et al.Real-Time Detection of Encrypted Thunder Traffic Based on Trustworthy Behavior Association[M]// Springer .Trustworthy Computing and Services. Heidelberg :Springer, Berlin, Heidelberg, 2012: 132-139.
[38]	QIN Tao, WANG Lei, LIU Zhaoli, et al.Robust Application Identification Methods for P2P and VoIP Traffic Classification in Backbone Networks[J].Knowledge-based Systems, 2015, 82:152-162.
[39]	GAO Changxi, WU Yabiao, WANG Cong.Encrypted Traffic Classification Based on Packet Length Distribution of Sampling Sequence[J]. Journal on Communications, 2015, 36(9):65-75.
	高长喜, 吴亚飚, 王枞. 基于抽样分组长度分布的加密流量应用识别[J]. 通信学报, 2015, 36(9):65-75.
[40]	SUN Guanglu, XUE Yibo, DONG Yingfei, et al.An Novel Hybrid Method for Effectively Classifying Encrypted Traffic[C]//IEEE. 2010 IEEE Global Telecommunications Conference GLOBECOM 2010, December 6-10, 2010, Miami, FL, USA.NJ:IEEE, 2010: 1-5.
[41]	HE Jie, YANG Yuexiang, QIAO Yong, et al.Fine-grained P2P Traffic Classification by Simply Counting Flows[J]. Frontiers of Information Technology & Electronic Engineering, 2016, 16(5):391-403.
[42]	CALLADO A, KELNER J, SADOK D, et al.Better Network Traffic Identification through the Independent Combination of Techniques[J]. Journal of Network and Computer Applications, 2010, 33(4):433-446.
[43]	BERNAILLE L, TEIXEIRA R .Early Recognition of Encrypted Applications[C]//PAM. International Conference on Passive and Active Network Measurement, April 5-6, 2007, Louvain-la-Neuve, Belgium. Heidelberg :Springer, Berlin, Heidelberg, 2007:165-175.
[44]	DRAPER-GIL G ,LASHKARI A H , MAMUN M S I , et al.Characterization of Encrypted and VPN Traffic Using Time-Related Features[EB/OL]. .
[45]	DING Rusheng.Classification of Encrypted Traffic Application Service Based on Spark Platform[D]. Beijing:Beijing University of Posts and Telecommunications,2017.
	丁汝盛. 基于Spark平台的加密流量应用服务分类[D].北京:北京邮电大学, 2017.

加密协议所在协议层	代表性加密协议
网络接口层	L2TP
网络层	IPsec
传输层	SSL/TLS
应用层	SSH、BitTorrent、Skype

识别对象	研究目标	主要特征
流级	流的特征、到达过程	流持续时间、流字节数等
分组级	数据分组的特征、到达过程	分组大小分布、分组到达时间间隔分布等
主机级	主机间的连接模式	连接度、端口数等
会话级	会话的特征及到达过程	会话字节数、持续时间等

识别类型	说明
加密与非加密流量识别	识别哪些流量是加密的,哪些是未加密的
加密协议识别	识别加密流量使用的具体加密协议,如SSL/TLS、Ipsec、SSH、Skype等
加密应用识别	识别加密流量所属的应用程序,如微信、迅雷或优酷;也可以是更深层的精细化识别,如微信里的消息、语音、视频或文件
加密服务识别	识别加密流量所属的服务类型,如网页浏览、流媒体、即时通信或云存储
异常流量识别	识别出是DDoS、PortScan、FTP-Patator、Bot、APT等恶意流量或攻击流量^[14]

识别方法	说明	识别内容	研究文献
基于有效负载的方法	从加密协议密匙协商过程中未加密的数据流里提取有用信息来识别协议或应用	部分未加密负载	文献[15,16,17]
基于分组负载随机性检测的方法	根据每个数据分组携带的一些相同特征字段的随机性来识别加密流量	负载随机性	文献[6]、文献[18,19]
基于机器学习的方法	加密技术不处理流统计特征,基于流统计特征的机器学习识别方法受加密影响相对较小	流统计特征	文献[3]、文献[20,21,22,23,24,25,26,27,28,29,30,31,32]
基于主机行为的方法	从主机的角度粗粒度地分析不同应用的行为特征	主机行为	文献[33,34,35,36,37]
基于数据分组大小分布的方法	根据数据分组大小分布的差异来识别,该方法受加密影响较小	数据分组大小	文献[38,39]
基于多策略融合的方法	大部分方法只对指定加密协议有效,将多种识别方法结合,实现高效识别	多种特征	文献[40,41,42]

流量类别	流量内容
E-mail	E-mail, G-mail(SMPT, POP3, IMAP)
VPN-E-mail	E-mail, G-mail(SMPT, POP3, IMAP)
Chat	ICQ, AIM, Skype, Facebook, Hangouts
VPN-Chat	ICQ, AIM, Skype, Facebook, Hangouts
Streaming	Vimeo, Youtube, Netfix, Spotify
VPN-Streaming	Vimeo, Youtube, Netfix, Spotify
File transfer	Skype, FTPS, SFTP
VPN-File transfer	Skype, FTPS, SFTP
VoIP	Facebook, Skype, Hangouts, Voipbuster
VPN-VoIP	Facebook, Skype, Hangouts, Voipbuster
P2P	uTorrent, Bittorrent
VPN-P2P	uTorrent, Bittorrent