Automatic Discovery And Display Of Hotspot Information On Web News

With the rapid development of the Internet, network has become the "fourth largestmedia" after newspapers, radio and TV. Because of the characteristics of openness, quicknessof information dissemination, the network has become the main carrier of informationtransmission, as well as the important source to produce hot spots. Therefore, how toautomatically discover the hot issues and the entity from the huge network resources has anextremely important significance.Hot information including hot words and hot topics, it associates with social events orobjects of significance which people generally concerned about in a certain period of time. Inthis paper, through the studying of existing technologies of topic detection and key wordsautomatic extraction, we discover the hot information from network reports automatically.The main works in this paper are as followed:1、Information crawl and pretreatment of web pages. In this paper, according to thefeature that website editors store web pages based on directory structure, it designs a crawlerbased on directory structure to extract web text. After that, it will do some pretreatment to thetext information, such as word segmentation, feature extraction, which can provide data todiscover hot spots.2、 In this paper, it proposes a sparse vector compression storage and similaritycalculation method in allusion to high dimensionality vector.3、 Automatic discover functionality of hot spots. First of all, it analyzes thecharacteristics of hot words and hot topics, and then uses word segmentation tool of Chineseacademy of sciences, ICTCLAS, to extract named entities while it extracts unnamed entitieswith the mutual information evaluation method combined with PAT-Tree. Finally, it carriesout heat evaluation and then extracts hot words. It uses Single-Pass secondary clusteringalgorithm to detect and track topics. Firstly, it performs local clustering to the reports of thecurrent day, which will form a candidate topic set, and then uses the candidate topic set to doincorporated clustering with the old topic set, the result will form a final topic set. After that,it performs heat sort to the final topic set and forms the hot topics in the end. 4、Display functionality of hot spots. After statistical analysis of hot words and hot topics,it displays the results in various chart modes.