Design And Implementation Of Mulit-Cores Parallel Indexing Solution Based On Lucene

Theory of parallel computing has been popularized widely due to great development of multi-core processor technology. This popularization strongly facilitates software performance enhancement through software and hardware. This paper proposes one design and implementation solution for multi-cores parallel indexing based on Lucene through analysis and study on parallel computing technology. This method conquers the bottleneck problem faced by Lucene, a full-text search engine when used for indexing large data sets.The parallel indexing proposed in this paper integrates traditional indexing process of Lucene through multi-cores parallel design. This program is based on mature parallel computing theory, supported by open programming interface of Lucene and parallel programming language, and make used of working pool and producer-consumer patterns.This paper presents the design and implementation of multi-cores parallel indexing solution in detail. The design of architecture and major core modules are also presented. The construction of working pool, parallelization of indexing buffer, and design and implementation of concurrent queue are illustrated.The innovative significance of this study is listed as follows. This paper combines the indexing of Lucene and parallel computing theory to implement the parallel indexing process, making full use of the hardware advantages of multi-core processors. Meanwhile, the design solution could enhance the indexing performance and then increase the work efficiency, which could decrease the use-cost and improve market competitiveness.