A High-speed Network Flow Reassembly Optimized Scheme for Network Security Analysis

doi:10.3969/j.issn.1671-1122.2019.11.011

Abstract

Abstract:

In high-speed network environment, network traffic collection and reassembly is an important prerequisite for network security analysis. To meet the need of the accuracy and real-time requirement of network security analysis, a high-speed network flow reassembly optimization scheme is proposed in this paper. Firstly, a parallel mechanism of multi-flow tables is designed in the Hash-based flow table scheme, the load balancing problem of high-speed network flows distributed among multiple flow tables is solved by introducing feedback information into the distribution strategy of high-speed network flows. Secondly, in order to further reduce the overhead of flow aging detection, an active queue is designed in the flow table scheme. Records are arranged in the order of least recent usage, which could avoid full flow table traversal operation and reduce the time complexity of flow aging detection. Finally, a high-speed network flow reassembly system based on flow table optimization scheme is implemented by DPDK, and the accuracy and real-time performance of the flow table optimization scheme are verified. The experimental results show that when the network bandwidth is 10 Gbps, the packet loss rate is 0.002%, which can effectively meet the data requirements of network security analysis in high-speed network environment.

Key words: security analysis, flow reassembly, multi-flow table, active queue, load balancing

CLC Number:

TP309

Liangguo CHEN, Shuhua RUAN, Xingshu CHEN, Yonggang LUO. A High-speed Network Flow Reassembly Optimized Scheme for Network Security Analysis[J]. Netinfo Security, 2019, 19(11): 82-90.

Figures/Tables 10

References 18

[1]	INTEL DPDK. Programmer’s Guide[EB/OL]. , 2019-6-1.
[2]	BLOTT M, ELLITHORPE J, MCKEOWN N, et al.FPGA Research Design Platform Fuels Network Advances[J]. Xilinx Xcell Journal, 2010, 4(73): 24-29.
[3]	ENDACE. DAG Packet Capture Cards[EB/OL]. , 2019-6-1.
[4]	HAN S, JANG K, PARK K S, et al.PacketShader: A GPU-Accelerated Software Router[J]. ACM SIGCOMM Computer Communication Review, 2011, 41(4): 195-206.
[5]	LEE J, LEE S, LEE J, et al.FloSIS: A Highly Scalable Network Flow Capture System for Fast Retrieval and Storage Efficiency[C]//USENIX Association. Usenix Conference on Usenix Technical Conference, 2015, Santa Clara, CA. Berkeley: USENIX Association, 2015: 445-457.
[6]	NTOP. PF_RING:High-speed Packet Capture, Filtering and Analysis[EB/OL]. , 2019-6-1.
[7]	DERI L, CARDIGLIANO A, FUSCO F.10 Gbit Line Rate Packet-to-Disk Using N2disk[C]//IEEE. 2013 IEEE Conference on Computer Communications Workshops, INFOCOM WKSHPS 2013, April 14-19, 2013, Turin, Italy. New York: IEEE Computer Society, 2013: 441-446.
[8]	EMMERICH P, PUDELKO M, GALLENMÜLLER S, et al. FlowScope: Efficient Packet Capture and Storage in 100 Gbit/s Networks[C]//IEEE. 2017 IFIP Networking Conference and Workshops, IFIP Networking 2017, June 12-16, 2017, Stockholm, Sweden. New York: IEEE, 2017: 1-9.
[9]	CLAFFY K C, BRAUN H W, POLYZOS G C.A Parameterizable Methodology for Internet Traffic Flow Profiling[J]. IEEE Journal on Selected Areas in Communications, 1995, 13(8): 1481-1494.
[10]	ZHANG Guangxing, QIU Feng, XIE Gaogang, et al.An Efficient Representation of Network Flow Record[J]. Journal of Computer Research and Development, 2013, 50(4): 722-730.
	张广兴,邱峰,谢高岗,等.一种高效的网络流记录表示方法[J]. 计算机研究与发展,2013,50(4):722-730.
[11]	PAPADOGIANNAKIS A, POLYCHRONAKIS M, MARKATOS E P.Stream-oriented Network Traffic Capture and Analysis for High-speed Networks[J]. IEEE Journal on Selected Areas in Communications, 2014, 32(10): 1849-1863.
[12]	NTOP. nProbe: An Extensible NetFlow v5/v9/IPFIX Probe for IPv4/v6[EB/OL]. , 2019-6-1.
[13]	INACIO C M, TRAMMELL B.Yaf: Yet Another Flowmeter[C]//USENIX. LISA’10: 24th Large Installation System Administration Conference, November 7-12, 2010, San Jose, California. Berkeley: USENIX Association, 2010: 107.
[14]	WANG D, XUE Y, DONG Y.Memory-efficient Hypercube Flow Table for Packet Processing on Multi-cores[C]//IEEE. 54th Annual IEEE USA Global Telecommunications Conference Energizing Global Communications, GLOBECOM 2011, December 5-9, 2011, Houston, TX, USA. New York: IEEE, 2011: 1-6.
[15]	ECKHOFF D, LIMMER T, DRESSLER F.Hash Tables for Efficient Flow Monitoring: Vulnerabilities and Countermeasures[C]//IEEE. 2009 IEEE 34th Conference on Local Computer Networks, LCN 2009, October 20-23, 2009, Zurich, Switzerland. Los Alamitos: IEEE Computer Society, 2009: 1087-1094.
[16]	BANERJEE S, KANNAN K.Tag-in-tag: Efficient Flow Table Management in SDN Switches[C]//IEEE. 10th International Conference on Network and Service Management, CNSM 2014, January 17-21, 2014, Rio de Janeiro, Brazil. New York: IEEE, 2015: 109-117.
[17]	WOO S, JEONG E, PARK S, et al.Comparison of Caching Strategies in Modern Cellular Backhaul Networks[C]//ACM SIGMOBILE. 11th Annual International Conference on Mobile Systems, Applications, and Services, MobiSys 2013, June 25-28, 2013, Taipei, China. New York: ACM, 2013: 319-332.
[18]	WANG Yucong, CHEN Xingshu,LUO Yonggang,et al.NTCI-Flow: A Scalable High-speed Network Traffic Processing Framework[J]. Advanced Engineering Sciences, 2017(S1): 168-174.
	王煜骢,陈兴蜀,罗永刚,等. NTCI-Flow:一种可扩展的高速网络流量处理框架[J]. 四川大学学报(工程科学版),2017(S1):168-174.

编号	CPU			内存	硬盘
编号	型号	主频	核数	内存	硬盘
1	Intel Xeon E5-2609 v2	2.50 GHz	8	128 GB	6T
2	Intel Xeon E5-2609 v2	2.50 GHz	8	128 GB	1T

用例模式	流表个数	是否使用流表并行化	是否使用活跃队列
1	1	否	是
2	2	是	否
3	2	是	是