Fast Snippet Generation Approach Based On CPU-GPU Hybrid System

As an important part of searching result presentation, query-biased document snippetgeneration has become a popular method of search engines that makes the result list moreinformative to users. Generating a single snippet is a lightweight task. However, it will bea heavy workload to generate multiple snippets of multiple documents as the searchengines need to process large amount of queries per second, and each result list usuallycontains several snippets. In order to improve the performance and economic efficiency oflarge-scale snippet generation under heavy workload, we propose a new high-performancesnippet generation approach based on CPU-GPU hybrid system.Firstly, I purposed a new snippet generation algorithm that suitable for GPU. Thisalgorithm adopts a sliding window style document segmentation, this method can avoidthe defect that may cut off the high relevance fragments caused by traditional approach.And this algorithm uses a new relevance calculation to score a fragment against query.Secondly, based on the analysis of the characteristics of CPU-GPU hybrid system, Iparallelized the serial algorithm in tasks and cross tasks. In order to achieve the optimizedefficiency of system, I designed a3-level process pipeline. I implemented it usingJobFlow architecture which supports service based programming. This architecture cansupport system design with high modularization and parallelization.Finally, I carried out a set of experiments to optimize the performance of the pipelinesystem, and verify the performance and economic efficiency of this system againstbaseline. The experimental results show that our approach gains a speedup of nearly6times in average process time compared with the baseline approach-Highlighter.