HTTP Flow Cobbling Algorithm Based On Deep Flow Inspection

HTTP flow cobbling refers to identify HTTP request containing access page’s URL from HTTP traffic which generated from web browsing. TCP flow it belongs to is defined as the main flow and HTTP request used to access embedded webpage object is in the auxiliary flow which is associated to the main flow. HTTP flow cobbling can be applied to the field of reverse billing, network fault location, web traffic analysis etc.Traditional HTTP flow cobbling algorithm has some disadvantages, such as HTTP flow cobbling based on IP and think times cannot be applied to HTTP traffic generated by the accelerated technology of CDN; HTTP flow cobbling based on referer and think times need to analysis the content of HTTP request and cannot deal with encrypted traffic; active HTTP flow cobbling algorithm which depends on the browser plug-in can only be used as an auxiliary method. This paper proposes a HTTP flow cobbling algorithm based on DFI which is recognition technology based on the flow behavior. It does not need to analysis the content of the application layer and can achieve HTTP flow cobbling through TCP layer. The algorithm has high efficiency and can be applied to HTTPS traffic.This paper researches the impact of Internet advertising traffic and CDN accelerating technology on HTTP traffic and proposes a HTTP flow cobbling algorithm based on HTTP request, studies the features of HTTP traffic by the algorithm and proposes a HTTP flow cobbling algorithm based on DFI by analyzing the features. The main works of this paper are:1) traditional HTTP flow cobbling algorithm based on DFI cannot process HTTP flow accelerated by CDN and Internet advertising traffic can impact main flow identification of HTTP flow cobbling algorithm, therefore, this paper researches the effects of CDN accelerated technology and Internet advertising traffic on HTTP flow cobbling and puts forward the corresponding solutions, which can achieve good results; 2) as manually collecting and labeling HTTP traffic lead to low efficiency, this paper proposes a method that automatically simulates behaviour user operating browsers based on Selenium and does a complete system which can access bulk URLs and collect corresponding HTTP traffic; 3) according to the characteristics of the HTTP protocol, we propose a flow cobbling algorithm based on HTTP request and realize HTTP flow cobbling by this algorithm for studying the characteristics of HTTP traffic; 4) according to the above work, we propose a HTTP flow cobbling algorithm based on DFI, and compare with flow cobbling algorithm based on HTTP request to verify the correctness of DFI algorithm.