Research And Implementation Of Web Content Monitoring System

With the rapid development of the internet,internet security monitoring has become an important task,according to statistics,most of the network traffic is web brower.thus web content monitoring becomes the main part of internet monitoring.This thesis is an implement of web content monitoring system based on ETSI standard. This thesis includes following parts1)The design of web content monitoring system,mainly include system architecture design,feature modular design and workflow design. The main function modules includes deep packet inspection module(identify web browse flow from online traffic)?protocol parse module(parse the data sent from deep packet inspection module,HTTP protocol parse thread will be loaded by protocol parse module in web content monitoring system,gain web content by reproducting the scene of web browse),data process center module(real-time forwarding the structured data to the database or query data from database in web content monitoring syatem),File process module(real-time storing the files to the file system or query files from file system in web content monitoring system),HTTP protocol parsing module is the foundation core part of the system.2)The detail design and implementation of HTTP protocol parse module in this system,include the relationship between HTTP protocol parse module and other modules,the feature and improve the search performance design(and the idea of the design) of this module . The design of internal structure,work process and main program flow of HTTP protocol parse module.Propose the improved ACBM algorithm used in the process of parsing downlink response packet inorde to reducing memory useage in system.The development environment is using C language in Visual Studio 2008 on Windows 7 64 bit machine,the compile tool is gcc,the deployment environment is Linux 64 on 64 bit system.3)Describe the implementation of the HTTP protocol parse module unit test,including analysising the function test point,disintegrating test target,considering the boundary and robustness test,designing and excuting the unit test.Describe the environment and process of the integration test by analysising the interface between HTTP protocol parse module and other modules.The main result of this thesis is designed an improved ACBM algorithm—ACBM-HB algorithm used in parsing downlink response packe in HTTP protocol parsing modulet,in order to reduce the memory use in system.The system has been deployed in actual network for more than a year,the operating result shows that the system can effectively achieve the goal of web content monitoring,can promote the internet content monitoring.