The Analysis And Design Of The Spam Filtering System Based On URL Classification Technology

With the constant development of the Internet, E-mail has become an integral part in people’s life. However, the by-product of the development-spam, not only greatly occupied the limited storage, calculation and network resources, and cost enterprises and users much time to handle them, but also caused the loss of mail users’personal information and property. And it seriously threatens to the information security of the mail users. Along with the development of email filtering technology, the proportional of spam which use URL resistance filtration is increasing day by day to avoid filtering technology inspection. There is no significant difference in content between the spam and the legitimate mails, but the content which the URL link is pointing to, which is also the sender’s real intention, is illegal, malicious. For this kind of spam, traditional spam filter methods often feel helpless.Based on reading a lot of references and related material, the author designed a kind of spam filtering system based on the URL classification technology, which can effectively filter out this kind of spam and improve the spam filter accuracy and recall rate.The main works are the following:1. Researching and analysising the characteristics of traditional spam filter technology, points out their limitations. Analysising the new method of spam inverse filtering, and then puts forward a new filtering method which based on the URL classification technology.2. Starting with the requirement, the thesis designs a spam filtering system which based on URL classification technology. Through the design of various management and function module, the system not only has the function to filter the URL which takes from the mails, but also meets the system requirements of reliability, maintainability and data real-time synchronization.3. Detailed designing the core function module of the filtering system-the module of URL classification inquires, including URL database design, inquires the process design, etc. In order to improve the working efficiency of the filtration system, adding md5, red and black tree and so on use of the algorithm in inquires, and designing the consistency of the classification scheme for the URL database. All of these improving query efficiency and accuracy of the filtering system.4. To realize high availability to the system, the paper designed and implemented some relative modules, and tested some main modules for the high availability of system.5. To insure the whole filter system feasibility, the paper integrated the filter system which the paper designed and the traditional filtering system, completed the integration testing and analyzed the test results.