Research On Topic Tracking System Based On Keywords

As a new direction of research on natural language processing, topic tracking which is a subtask of TDT, airms at monitoring the stream of news stories to find additional stories on a topic that was identified using several sample stories. The research on topic tracking in China is just starting.In this paper, studies on topic tracking in Chinese news stories are carried on. Several algorithms for topic tracking are proposed and improved. Meanwhile, unigram model for Chinese topic tracking is adopted in the topic tracking system.The main work in this paper is as follows:Analysed the advantages and disadvantages of feature selection in text classification and then proposed a method of feature selection- the feature selection based on the frequency in category. This method was applied into topic tracking system, and achieved good results.TF*IDF weighting method, in text classification obtained a better results, but this method lacked of theoretical basis, and didn't take the feature proportion in the text set into consideration. Therefore, we proposed to add the value of feature assessment function in feacture selection to the weight calculation method and overcame the shortcomings of TF * IDF.In order to improve the performance of topic tracking system, we proposed unigram model for Chinese topic tracking and analysed some factors that affect topic tracking performance.We considered the part of speech in unigram model, proposed to use word pair as fesrures. However, this method can not achieve a better performance, but reduced dimension of features and improved system performance on the condition of not increasing the system overhead.