| With the rapid development of mobile communication technology and the rapid increase in mobile penetration, with the features of simple usage, fast delivery and low-cost SMS became an integral way of communication in our daily life. However, low-cost transmission makes us receive a lot of junk short message every day. These junk short messages are harmful to the social security and poison social atmosphere. What is more, it influences users' normal life. The current junk short message filtering technology includes black and white list technology, filtering rules and based on keyword matching filter etc.In order to solve the problems caused by junk short messages, the article firstly gives a detailed introduction and analysis on features, causes, damages and the current status of junk short message in our country, at the same time the article summarizes the current status of message classification techniques. Secondly, the article introduces 3 methods for dividing Chinese words,4 kinds of text representation,7 kinds of feature selection method,4 kinds of feature weight assignment method and 7 kinds text classification algorithm. Finally the article uses an improved Bayesian classification combining black and white list filtering mechanism to filter messages. The Message classification model includes message preprocessing, feature selection, weighting calculation, classifier training and message testing. It receives a good classification result, what is more, by using new training dataset, it can train personalized classifier to adapt the changes of short message and meet user's needs. The experimental results prove that the improved Bayesian classifier can reduce the error rate of normal SMS. In detail, this article contains the following main elements:1) Introduced several feature selection methods and making a comparison between their advantages and disadvantages in the classification model;2) For the current Chinese text corpus lack of publicity and standard, I collect and establish a standardized and what can meet the experimental requirements Chinese text corpus;3) On the basis of the short message text corpus I achieved an improved Bayesian classification model to increase the through rate of normal SMS. Then I analyzed and compared the model's performance and effect; 4) Classification model can produce personalized classifier to adapt the change of messages, what it can meet the needs of users;5) According to the evaluation standards of Text Classification I evaluated the Bayesian classifier from experiment. Then according to the evaluation, I set the basic parameters of the classifier model;6) This article summarizes the the results, outlooks the further works, for the future work to lay a foundation. |
