The Research On A Term Weight Calculation Method Based On The Term Mathmatical Expection

With the rapid popularization of the Internet and information technology, the information which we can gain becomes more and more.How ever,the information which people really need is difficult to find, while the various search engines can solve this problem to some extent, but now the various search engines are keyword-based search, which makes the search results is enormous, so do not need help users find accurate information. Text categorization is an effective method which is the core technology search engine technology. With a variety of machine learning methods and statistical theory is introduced into the text of the automatic categorization, automatic text categorization becomes a research focus in data mining.The high dimensional feature space and the sparsity automatic are difficult problems, So finding an effective term weighting method to reduce the dimension of feature space and improve the efficiency of text categorization and accuracy becomes the first problem to be solved. In this research , main analysis the shortcomings of the current TFIDF algorithm, first of all not considered the text number which contain the term distribution between the various categories, followed not considered in a category that contains entries of the text are distributed, Finally do not take into account the various categories of training documents, to decrease the shortcomings of traditional computingm, mathematical expectation term TFIDF-E algorithm is proposed.In the algorithm , the term as a random variable to weight the variable with the mathematical expectations,In the last,experimental results verify the effectiveness of the proposed method.