Resource efficient parallel VLDB with customizable degree of redundancy

Databases have become critical part of human lives. As their sizes multiply annually, their importance grows exponentially. Ironically, although database servers hold the vast majority critical data, they are the hardest to scale up and cost prohibitive to protect.;This thesis focuses on the practical use of very large scale relational databases. It leverages two recent breakthroughs in parallel and distributed computing: (a) synchronous transaction replication technologies by Justin Y. Shi and Suntain Song; and (b) Stateless Parallel Processing principle pioneered by Justin Y. Shi. These breakthroughs have enabled scalable performance and reliability of database service using multiple redundant shared-nothing database servers. In this thesis, we present a methodology with customizable degree of redundancy to address practical very large scale database applications problems.;This dissertation focuses on a resource-efficient customizable VLDB parallel replication system. It aims to deliver the following: (a) A customizable VLDB parallel partition and replication strategy that will allow automatic dataset partition, data distribution and seamless (non-stop) recovery. (b) An application programming interface that will automatically translate user queries using virtual global schema into parallel queries targeting multiple physically partitioned schemas. The central theme is to optimize overall efficiency of a VLDB application with variable degrees of redundancy.;The prototype VLDB implementation relies heavily on a prior work named DBx by Dr. Shi and Suntain Song. DBx is a general-purpose VLDB cluster runtime substrate capable of delivering high performance and high reliability at the same time using multiple shared-nothing database servers. It was designed for transparent non-intrusive deployments. Due to the critical nature of the subject matter, we believe our findings will be useful references to researchers in the database community in the large.;This dissertation focuses on a new data partitioning technology to allow a resource-efficient implementation and its supporting application programming interface. The prototype system supports Microsoft SQL Servers databases. Computational experiments are conducted using industry-standard benchmarks.