Keyword And Key Concept Extraction Technique Based On WEB Page

Keyword Extraction is an important technique of text information process. At present, Keyword Extraction is an important technique used for automatic abstract, automatic classification, subject extraction, subject word extraction etc. The paper introduces a new technique of keyword extraction and key concept extraction based on Web page, the design and implement of experimental system, and the application of the system in the search engine. The paper includes three main part.First, Keyword Extraction System. The paper describes the special of Web page compared with the common text. Depending on the special, a technique of keyword extraction based on Web page is introduced. The system takes full advantage of tags in the Web page.Second, Key Concept Extraction System. Language is a developing culture, and new concepts are produced. And many proper names which include person name, geography name and corporation name, are new unknown concept. These concepts have an impact on the result of Keyword Extraction system. The paper brings forward a key concept extraction technique based on the mutual information and context dependency. The means avoids the truncation effect of N-gram model and realizes vari-gram statistical model of concept extraction. At the same time, the paper adopts the way based on rules to optimize the extraction result.In the end, a simple research is done for the application of the system in the Search engine. By analyzing the relevance of search engine, the paper brings forward a improved system relevance model and describes the design of the model.