Research Of Focused Search Engine About Petroleum Subject

Information industry developed rapidly in the past few years, but there is little focused search engine about petroleum theme. Most of the existing researches are about the synthetically search engine and the researches about petroleum theme are imperfect. So it is significance to research the focused search engine about Petroleum Subject.Introduce the research actuality and working theory of the focused search engine. Expound the working theory and search strategies of focused spider. After study all the improving methods in the world, analyze the deficiency and then propose several improvements.Parse the text to build a DOM tree, propose a word co-occurrence model which bases on the DOM tree. Extend the preliminary topic dictionary by co-occurrence information. Compared with the original, expanded petroleum topic dictionary can guide the collection more comprehensive and effective.After consider a number of factors which impact the weight, analyze the TFIDF algorithm. Regard paragraph as a unit and calculate the similarity and improve N_Level_Vector Model consequently. Reduce the dimension and extract the theme of information more effectively.Propose an adaptive focused crawling strategy based on the context of reverse links. Using reverse links context to extend the topic dictionary, and then calculate the priorities of links by the topic dictionary. Optimize the Calculation of link priority in order to make the focused crawling strategy more efficiently. Experiments prove that the adaptive focused crawling strategy is better than the traditional strategy. It has no thematic excursion phenomenon, and achieves the adaptive effect.