Design And Optimization Of Primitive Execution Engine For Column Database

With the coming of information age, amounts of information that people are collectingare larger and larger. How to analysis and use the data with efficiency has become a problemof current information system nowadays. In the past years, a variety of new architectures ofthe microprocessor, especially the multi-core processors, had been developed with theprograss of integrated circuits and the computer architecture development. The heterogenouscomputing platform composed of multi-core CPU and GPU provides a powerful computingability, how to make full use of the potential computing resource of the heterogenouscomputing platform and improve the performance of database query is a key issue of databaseresearch.In this paper, a primitive execution engine base on column database is proposed. Underthe heterogenous platforms, after the SQL compiler transforms the statements into primitivedependency gragh, primitive execution engine dispatchs the database primitives into themulti-core CPU or GPU computing unit. The column database primitive means that theprimitives is a database operation type, which is sutable to column stroge, such as join, group,sort and so on.The data handled by this primitives stores as a vector, so it is in favour fordeveloping the parallel computing ability of multi-core CPU and GPU.This paper will study and solve the following problems:(1) Primitive execution mechanism. The primitive is classified into I/O primitive，computing simple primitive and computing complexity primitive.The primitive executionprocedure is driven by ready data,a level primitives finish executing,it push the next levelprimitives,until the last level primitives finish executing,the query complete.Befire theprimitive execute,schedule thread sleep,work threads are woken;after the primitive finishexecuting,work threads sleep,schedule thread is woken.(2) Dynamic primitive schedule method.In order to make load balance, this paperintroduce three dynamic schedule methods, which involve CPU priority, GPU priority andfixed type priority schedule method.(3) Static primitive schedule method. After the primitive dependency gragh isgenerated,engine set a execution path base on join primitive for GPU primitive.CPUprimitives are scheduled by schedule method base on dynamic programming and GPUprimitives are scheduled by schedule method base on GPU memory management machanism.(4) Test of typical complex queries. Database benchark test, TPC-H, includes8tablesand22queries. It is the internationally recognized benchmark program for complex queryies.In this paper, we will choose several queries to test and analysis with the primitive executionengine present in this paper.