Baidu Academic Oriented Real-Time Retrieval System Research

With the development of the Internet, a lot of information emerge in various media, hot information is paid more attentions to by people, and people want to get current hot issues at the fastest speed when they retrieve information. The design of the index structure in the traditional retrieval system can't support frequent data update, and rapid access to the updated data can't be gotten by users. The traditional retrieval system's ranking function does not take into account the similarity between the query contents and the query keywords, so that more accurate results will not be obtained by users.In order to solve the problem that the traditional index structure does not support frequent data update, an adaptive index structure is put forward in this article.This adaptive index structure adopts a read and write index combined mechanism and can adaptively increase its size with the insertion of datas. The adaptive index structure can improve the efficiency of the query, and support data update and query at the same time. It also does not affect the query efficiency in the process of update.In view of the traditional retrieval system does not consider the similarity between the query contents and the query keywords. A new ranking function is put forward in the article. The new ranking function not only considers the similarity between the query keywords and the query contents, it also considers the popularity of the query contents. Experiments proved that the new ranking function can improve the accuracy of the results returned by the query keyword.The test on the baidu academic oriented real time retrieval system showed that the system's updated data can be retrieved immediately, and it meets the requirement and characteristic of the real-time retrieval system. The comparision between the adaptive index structure designed in this system and the current popular segmentation index structure shows that the design of the adaptive index structure in this system had made at least 5% improvement in performance. The experimental results also show that the new ranking function can be more accurate, making the results' distribution more focused.