Implementation And Optimization Of Data Integrity Constraints For Scalable Database Systems

With the rapid development of big data and Internet technologies,the traditional centralized database systems have been difficult to meet the processing demands of new services for massive data and a large number of high concurrent workloads.As a result,the scalable database system has gradually become the most popular research topic in the field of the database,and a number of new architecture New SQL database systems have appeared in the market.Not only do they have powerful data storage and transaction processing capabilities,but they also have good usability and scalability.In relational database systems,four types of data integrity constraints are usually used to ensure the accuracy and validity of data.For traditional centralized database systems,both entity integrity and referential integrity constraints are mature and essential.However,for the new scalable database system,especially those New SQL database systems based on the log structure merge tree storage architecture,the implementation of these integrity constraints is not complete enough and challenging.Therefore,this paper starts with the data integrity problem in the database system,and deeply studies how to implement the entity integrity and referential integrity constraints in the scalable database system.In addition,for the architectural features of the scalable database system and specific application scenarios,this paper also explores a variety of optimization techniques to maximize the transactional performance of these two types of integrity constraints.The main contributions of this paper are summarized as follows:1.Aiming at the transaction characteristics of entity integrity and referential integrity constraint related functions,we summarize a class of implicit long transaction operations that are universal,and the detailed implementation algorithm of these implicit long transaction operations in the scalable database system is further given.2.We propose a series of optimization techniques such as lightweight primary key generation manager,adaptive batch processing mechanism and distributed parallel processing.They ensure that these integrity-constrained transaction operations still perform well in distributed,high-concurrency,and big data scenarios.3.We integrated the above implementation and optimization techniques in the open source scalable database system CEDAR,and further proved that these methods are correct and efficient through a large number of experiments.In summary,this paper studies how to implement and optimize data integrity constraints in a scalable database system.Firstly,a complete set of the entity and referential integrity constraint implementation schemes is presented for the architectural characteristics of scalable database systems.Secondly,a series of optimization techniques are explored to improve the execution efficiency of transactions in distributed and high concurrent environments.Finally,a lot of experiments prove the correctness and efficiency of these methods.