Research And Application Of Carrier-class Intelligent Network Information Filtering Engine

Aiming at the situation of lots of illegal information spread in present network, we implement an intelligent web information filter engine which is applied to the telecom main communication cable. By constructing category knowledge-base and using the intelligent information processing technique (natural language processing), the system is able to identify the content and filter measures based on the understanding of the content of the internet informatioin (not only matching the key words) and obtaining the meaning and the attitude of the information. To filter the information of telecom high speed network, we also designed and implement the module of high performance packet capturing and application layer protocol analyzing. The key techniques of the engine include the natural language processing technique, the high performance network techniques (protocol analyze and zero-copy technique), and so on.The main works finished in this paper are listed as follows:(1) By importing natural language processing theory, I implement intelligent URL category library based URL category knowledge-base. Using this we need not analyze and filter internet information at real time, but just create the URL category library before system starts. That makes it can be applied to filter information of high speed network.(2) The research and application of the zero-copy technique overcomes the traditional packet capture technique' s shortcoming. The performance makes filter high speed network' s information possible. We also can use normal devices to process high speed network information by using this technique which decreases the expense.(3) The module of analyzing application layer protocol was designed which is able to filter information by identifying the content but not by matching the key words.