Research On Topic Detection And Tracking Based On Probability Topic Model

With the emergence of the new media——Internet, there are a wide range of information sources. How to get information timely, accurately and objectively is become an urgent problem. Topic Detection and tracking provides a way to handle news, organize stories into topics and browse news events. The traditional information retrieval technologies have been widely used in TDT. In recent years, a probability topic model which based on graph theory and statistical theory has attracted research people's attention. Probability Topic Model is a typical non-supervised learning method. By introducing a latent variable (topic), topic model can generate a data set. The idea based on topic makes probability topic model is suitable for extracting topics from a data set. So we try to use topic model to do topic detection, topic tracking and topic organization. And use a series of experiments to verify its efficiency.First, we use Supervise Topic Model which is developed by David for data-label pair wise data set, to train the given story-topic pair wise data set. After model training, a reasonable topic model to interpret a story's label (topic property). Then we use DET cost curve to do the evaluation.Then, Dynamic Topic model is used to do topic detection with the fact that news stories are not exchangeable. But Latent Dirichlet Allocation is based on document exchangeability. After dynamic topic model training, we use the distribution over words of topics to check the similarity between topics, and use the distribution over topics of stories to interpret a story's topic attribute.Last we give a new conception of topic organization. It's based on dynamic topic model too, for the purpose of provide people with a way to browse news events from a more abstractive view on the topics.