| Network traffic analytics is a well-known core technique for network service providers capturing the network status,one precondition for network content providers accurately analyzing services.Besides,it plays a more important role on the network security and protection.Nowadays,the traditional network traffic analytics have to meet lots of problems brought from the increasing numbers and complexity of new network protocols under the fast Internet development.One of the problems is a mount of requirements on the one-by-one customized analysis tool for each new private protocol.Second,as one necessary algorithm of the network traffic analytics,the multi-mode matching becomes low performances and needs long update time due to the huge size of protocol rules,above 100,000 rules for example.Third,a type of high scalability analytics tools is required to smooth unbalance developments between the rapid grows of network traffic and limited improvements on the CPU performance,especially the main CPU frequency.To solve the three issues and hence improving the steam analytics performances,the models of the network traffic analytics process is investigated,the core algorithm of the multi-mode matching algorithm is improved,and the performances of multi-core platform and parallel processing on the steam analysis are optimized.The details are as follows:1)A configurable information identification and extraction algorithm is proposed and realized in a configuration engine to reduce the difficulty and workload of development on the network traffic analytics tool for every new protocol.The proposed algorithm is based on the investigation of numerous common information identification and extraction methods for standard and non-standard protocols.Then,it is used in a configuration engine realized by the configuration XML scripting language.The engine is able to identifying protocols,splicing messages,recognizing contents,and outputting metadata.2)The multi-mode matching algorithm is enhanced.The widely used ACBM algorithm,especially the bitmap compression-based ACBM algorithm,is helpless to dynamic update rules due to the low effectiveness of jump matching.Therefore,an enhanced AC algorithm is proposed.The new AC algorithm adopts the idea of the Sunday algorithm jump rules and is able to real-time update the pattern rules.Test results show that the enhanced AC algorithm process above 10,000 matching per second;meanwhile,the average matching time per thousand packets is reduced to5.7ms.As a result,the proposed algorithm improves the matching performance by about 34%higher than the standard ACBM algorithm.3)A network traffic analytics tool with strong scalability is studied and completed on multi-core platform and reaches the aim of linear expansion of functions and performance.In the study,the characteristics of CPU under UNMA architecture,and parallelization technologies such as memory management,CPU affinity,and parallel pipeline architecture is investigated.Based on the investigation,the high-expansion parallel processing architecture is proposed and tested.The test results show that the architecture is linearly extended with the increase of the number of CPU cores.Finally,based on the above three research results,the network traffic analysis software toolkit based on the application layer dynamic feature recognition technology is implemented and tested.The toolkit implements configuration identification analysis of the mainstream L2-L7 layer protocols,and its performance also reaches 10 Gbps. |
