Research And Implementation Of Data Synchronization Middleware For Heterogeneous Relational Database Based On Zookeeper

With the continuous development of database technology,enterprises in different environments using the database may be different,basically are relational database,how to solve the different systems to access the data between the database agreement is particularly important in the actual environment.Therefore,the establishment of a common data synchronization middleware system has become the main technical way to solve the problem of data synchronization between heterogeneous database systems.Although some of the current data synchronization methods are many,but these methods have their own advantages and disadvantages,it is difficult to specifically meet the needs of the enterprise environment,real-time requirements in the environment is difficult to apply,while most of these methods are ignored The concept of transaction in the heterogeneous database,when the problem occurs easily lead to data inconsistencies.In view of these problems,this paper designs and implements a heterogeneous relational database synchronization middleware system based on Zookeeper.The main work is summarized as follows:In order to solve the problem of high resource occupancy and poor universality in the process of changing the acquisition method in heterogeneous database synchronization,a method based on event notification and the method of changing capture uses log is proposed to avoid the log training program.The method is suitable for Oracle database changing capture in this paper's environment,the system resource overhead is small.In order to achieve the synchronization function,log capture based on Zookeeper distributed service components and message queues ensures the order of the message consumption,there will be no synchronization statement disorder at the time of execution.In the process of heterogeneous database synchronization,due to the heterogeneity of each database product data structure and data type,this paper proposes a method based on SQL restoring method to transform the captured SQL statement into SQL statement corresponding to the database platform.This method ensures the openness and scalability of the system.In order to test the solution proposed in this paper,a heterogeneous database synchronization middleware system with configurable and high performance is implemented.Use the program to simulate data changes,and synchronize these changes to other databases,test and analyze the results,find the factors that affect the system performance and shortcomings.The experimental results show that the system can fully guarantee the autonomy and integrity of the database system,the impact on the performance of the database system is low,it can be stable and reliable to complete the synchronization between heterogeneous databases.