Audio And Video Resources Online Traceability For Mass Flow Key Technology Research

As an open platform, there are a series of web service allowing users to share the resources to others on the internet. Cyberlockers(one click downloading resources) website, such as network disk, resources sharing website has become the most important way to share because of its simple operation, download speed and other characteristics. When CLS users click the share link to download the sharing resources, the user's browser automatically sends a series of HTTP sessions, until the establishment of HTTP session transporting resources data, in this paper, we define it as URL chain. URL chain must pass through the backbone network nodes, how to obtain the share links to download audio and movie resources HTTP session from the big network traffic of backbone nodes for Internet censorship backbone network traffic,network forensics, network traffic monitoring, movie copyright protection, complex network environmental information content to uniquely identify, massive traffic aggregation and correlation of data link, network flow object extraction has very important significance.To this end, we propose a solution for online resources traceability—cookie Tracking based on tracing the cookie of downloading resources user. This solution firstly identifies HTTP sessions transmitting movie resources from the big traffic flow of backbone network nodes and uses the accumulated hash algorithm for arriving resources packet fragment in order to calculating the resources ID; then, according to the bill of HTTP cookie similar analysis, using the inverted index methods to grapping resources URL chain; finally achieve the ultimate resources traceability by extracting the unique common node from several URL chain of the resources.Further, in this paper, we developed a cyberlocker resource traceability system—cookie Tracing, for the backbone network node. The system uses a distributed flow approach, the front-end capture TCP flow based on traffic flow capturing platform, parse HTTP message required, identify resources HTTP sessions, calculate the resource ID, and sent to the back-end server through POSIX socket API and Google Protocol Buffer; the back-end are responsible for cyberlocker resources traceability: Firstly, using the Reactor pattern of event handing design, the main thread placed the message sent by HTTP front-end in the HTTP message queue, multiple worker threads construct the associate HTTP session with cookie and location by hash table; Secondly, the resources traceability thread obtain the URL chain by tracking the crucial cookie using inverted index method, the share link seeking thread achieve resources traceability by combining a plurality of corresponding URL chain to unique common URL; Finally,using the web crawler program access the share link,download the resource and calculate the resource ID value, compare the coherence to verify the correctness of traceability system. In addition, cookie Tracking zero-copy technology to front-end data packet receive and transmit, the number of worker threads dynamic adjustment mechanism, the thread poll mechanism to optimize system performance.The experimental results demonstrate the effectiveness and extensibility of cookie Tracking method, at the environment that bandwidth is 2Gpbs, running continuously 5*24h, where the average precision rate, recall rate, access time respectively has been reached 85%, 70%, 5000 ms.