Research On Approach To Building Text Index Based On GPU

With the rapid development of Internet, web information increases dramatically everyday. Due to the large amount of data and computation, it costs a lot of time for searchengines to index the large large-scale text, which reduces the searching performance. Sothere is a need to propose a fast indexing approach to improve the efficiency of real-timesearch.In this thesis, a fast indexing approach for large-scale text is investigated by combingthe technology of GPU parallel computing and text indexing. First, a collaboration modelis introduced for high-performance parallel text indexing. In this model, GPU is in chargeof index construction and the CPU is responsible for controlling. A pipeline work mode isfurther designed to make full use of the power of both CPU and GPU. To improve theefficiency of parallel computing, an index structure that suits the memory structure ofGPU is presented. We design two index structures which are based on hash table and arrayrespectively, and then a management strategy for index data is designed for large-scaletext indexing. Then parallel computing algorithm for indexing is proposed based on theindex structure. In detail, an approach for duplicate removal and frequency counting inparallel is designed, and the parallel algorithm for forward index inversing and merging isfurther designed. To further improve the performance of indexing, several optimizationstrategies are carried out according to the characteristics of GPU parallel computing. Theexperimental results show that our scheme can effectively exploit GPU and GPU canimprove the performance of large-scale text indexing.