Research And Implementation For Heterogeneous Data Transformation And Synchronization Technology

At information ages, it is an important index of the development potential that whether enterprises obtain new, accurate and comprehensive information or not. The data integration being responsible for the data to flow toward the target database from the data source is an important method that enterprises obtain information, is the important foundation that accomplish the data integration and share, is the core that building the data warehouse, is the premise of decision analysis support system.This paper designs and implements a data integration system, with focus on key technologies in data integration, namely, data integration system structure, heterogeneous data transformation methods and heterogeneous data synchronous strategies.In the process of data integration system structure research, according to analyze the currently traditional data integration structure, we put forward a three-layer proved data integration system structure on base of the module ideas in the software engineering. In this structure, the data integration stages are divided into three parts of independence. The logical business of data transformation is separated, and it gives satisfaction for the need on the distributed environment.In the research of heterogeneous data transformation methods, according to analyses and compare many transformation methods, we put forward a data transformation method basis of metadata-driven. In this method, all information are regarded as metadata and saved in metadata center. All transformation and cleaning are performed and controlled by metadata and customers can register transformation functions toward the system. Except that, the reusable mechanism is provided and some public transformation methods and strategies can been used by another customers.In the research of heterogeneous data synchronously strategy, we analyze the synchronous strategy basis of snapshot differential algorithms and the synchronous strategy basis of log check. According to analyses and compare snapshot differential algorithms, we seize the apply scope, handling speed and accuracy. Then, we analyze the synchronous strategy basis of log check. Aim at above-mentioned strategies low efficiency, implementation complex, and according to the triggers support degree and some characteristics under distributed network environment, we put forward two kinds of synchronous strategies: synchronous strategy basis of trigger and trigger, synchronous strategy basis of MD-5 algorithm. In the synchronous strategy basis of trigger and trigger, we combine trigger and trigger together and reduce the monitor object from row to fields, reduce data redundancy on network. In the synchronous...