JTangSpider:The Design And Implementation Of A High-performance Oracle Database Synchronization Platform

In recent years, more and more enterprises have developed business with internet, and have offered service in different regions and domains, so distributed database system has become the foundation of the enterprise. With the development of business, enterprises need to integrate different and isomerous databases and synchronize incremental data real-time. For the demand, this paper proposes an Oracle database synchronization architecture model with high stability, expansibility and compatibility, which is composed of incremental data extraction, data caching and processing, synchronization mapping and message routing.In order to solve the problem that traditional database synchronization middleware doesn’t support different version of Oracle database, this paper extracts incremental data with trigger, LogMiner, local CDC and remote CDC, and adopts different methods according to the version of database and special requirements.In order to solve the problem that traditional database synchronization middleware can’t recover errors quickly, can’t update the logic of data processing and can’t expand the performance, this paper designs a new cache model with memory and MySql database and manages handlers of data processing with the Chain Of Responsibility. What’s more, it implements an adaptive control system and recovery system.In order to solve the problem that there are kinds of synchronization mapping modes and the environment of network is complex, this paper designs the mapping manager and message router. The mapping manager overcomes the difficulty of managing different mappings by dividing mappings into three kinds:one to one, one to many and many to one. Message router takes charge of sending and accepting data for users.In order to solve the problem that it is difficult to adjust the contradiction between data sharing and isolation, ensure the data security and implement load balancing, this paper proposes a hierarchical mechanism of roles and a classification method of data. Different roles have different permissions and different data is stored by different ways. What’s more, this paper designs a measurement mode of load for service nodes and develops an algorithm to keep load balancing among service nodes.This paper develops a prototype system and tests the performance of data extracting and data processing. The result shows the system support three kinds of DML operation:insert, update and delete. What’s more, the system can expand the ability of data processing according the amount of data.