Research On Self-organizing Tuple Reconstruction For In-memory Column Database

With the increasing amount of data in the database, performance issues becomemore important than ever. Because column-oriented database loads data faster, onlyread the relevant attributes, has a high compression ratio, it has been widely used inthe field of data analysis. However, it must involve connections of more than onecolumn for constitution of the output tuple, which is called tuple reconstructionbecause of storage model of column-oriented database. The tuple reconstruction isthe most time-consuming step in the column-oriented database operations. Therefore,reconstruction of the tuple has been one of the hot topics in the field ofcolumn-oriented database.As a self-organizing tuple reconstruction method, partial sideways cracking hasa low adjustment cost and can adapt itself to the the dynamic environment in asustained incremental adjustments based on workload without human intervention.These characteristics can well meet the increasingly complex work environment.Nevertheless, as we show, existing partial sideways cracking methods fail to deliverworkload-robustness. It just blind to adjust the environment based on the query,however, this adjustment may need to pay a higher adjustment costs with a littleeffect. In this paper, we introduce sorted robust partial sideways cracking, asignificantly more robust approach. Sorted robust partial sideways cracking alsouses each query as a hint on how to reorganize data, but not blindly so. And it canfurther speed the tuple reconstruction in column-orientd database. Experimentsshow that sorted robust partial sideways cracking get a breakdown in robustness andspeed of the tuple reconstruction.(1) Studys and analyzes the problems in the partial sideways cracking. Analyzespartial sideways cracking robustness issues and alignment operation cost problem ina different workloads through theoretical and experimental analysis.(2) Based on partial sideways cracking, we introduce the SR partial sidewayscracking (sorted robust partial sideways cracking) for some robustness problems andalignment cost issue encountered in partial sideways cracking. And we proved itsrobustness and efficiency through experiments.(3) Implements a range query database system based on SR partial sideways cracking, lays the foundation for future studies based on SR partial sidewayscracking.