Bad Text Filtering System Research And Implementation

With the development of Internet, we are enjoying the convenience and efficient brought by the network。At the same time, based on a variety of bad information (erotic, reactionary, violent information and so on) are also more and more interference with the normal order of the Internet, and affecting people’s life. How to effectively control the spread of those bad information, guarantee network security, has become a very important problem.Based on the information filtering technology research, this paper introduced predecessors’ work about the bad information filtering system particularly. First it discussed the relationship between text filtering, information filtering, and information retrieval. This paper also discussed the classification of information filtering system and summarized the current research situation about information filtering system at home and abroad. Then it did some research into the Chinese text filtering technology, such as information filtering model and evaluation method, HTML tags, word segmentation, filtering processing stop-word, user template set, string matching algorithm, and mainly introduces the KMP, BM and Bloom Filter algorithm.Finally, This paper presents a bad text filtering system based on Bloom Filter The system directly analyse real-time network information, then shows the suspect URL to analysts through the web browser, at the same time analysts can also customize the keywords needed to filter and their weights through the web browser.