Research And Implementation Of The Bitmap Index In Column-Oriented Data Warehouse

With the explosive growth of the datasets, the hotspot of the database research has changed from the online transaction processing to online analysis processing database. Database has been taken as the most popular technology for the huge datasets analysis, which makes great effort to the enterprise decision, and widely been paid attention to the recent research. Huge data storage and ad-hoc query are the two core subjects in the database research field.There are three common ways to improve the huge data query performance. First of all, we can change the data storage model to better meet the data query. Secondly, we can create effective indexes to improve the query performance. At last, with the help of the query optimization technology, we can make great improvement of the data processing, such as the equivalent transformation, query rewrite and so on.Huge theory and practice have proved that column-store system is superior to the row-store system when face to the analysis application for the huge datasets. In order to make a great improvement of the query performance, DWMS adopt colum-store technolgy.Index is another core technology to improve the query performance, especially for the column-store system. The current paper focused on the bitmap index and its vector compress method, as a research results, the current paper proposes an efficient bit vector compression algorithm and implements the bitmap index on column-oriented DWMS system.The current paper makes a deep research on the bitmap index technology, especially for the bit vector compression algriothm. As to the shortage of bit vector compression technology, the current paper propose a new bit vector compression method, which called adaptive partition word-aligned hybird bit vector compression technology(APWAH). APWAH not only can reduce the size of the bitmap index, but also can make great use of the CPU operation feature. We also implement the logical operation on the compressed vector directly, which reduce the uncompress cost. At the same time, since the OLAP queries will produce large intermediates when execute the whole query, so we can choose the compressed vector to stand for the position list of the intermediates. In this way, we can save the space in the data buffer, and reuse the intermediates as efficiently as possible.At last, as for the algorithm we proposed, we design some experiments. The experimental results on the benchmark data sets SSB verify the effectiveness of the proposed method.