Middleware-Based Heterogeneous Data Integration

Because of historical reasons, the data form participated in exchanging in the application is varied. Therefore the way it is visited is also various. Especially since relational database products come out, people pay close attention to the issue of data interoperation and data sharing.Data integration mainly solves the problem of ensuring interoperability between heterogeneous data sources. Function involves accessing, transmitting, exchanging, transforming between the data sources and data synthesis, data replication, etc. Federated Databases and data warehouses are successful data integration solutions, but these solutions are not only complicated but also theme related. We consider more flexible solution of middleware-based heterogeneous data integration.The middleware is a big class of basic software between operating system and application. Because it offers the transparency of stepping the network and platform, and supports standard protocols and interfaces, the complexity of the data integration in the distributed environment can be reduced greatly.We design a middleware-based system, HeteroDI. It utilizes Internet to integrate heterogeneous data sources. Because it bases on ODBC interface to extract and load data sources, the majority of current relational databases can be integrated. Furthermore, it supports many kinds of channels. So the flexible of data transmission means is evident.Heterogeneous data synthesis is an important function of data integration. It involves data conflict resolution, data replication and more complex operation such as join, merge and difference. We present the basic operations when performing data synthesis in HeteroDI system, put forward the solution of the common data conflict, realize the data incremental replication based on trigger and snapshot.With the technology and the application of XML developed, it has already become an important information issue standard on Internet, and more and more applications exchange data through XML. The quality of stepping platform and strong adaptability of XML has been accepted by the middleware research field. We use XML DTD to define metadata and transmit package of HeteroDI system. So the expandability and independent character of HeteroDI are supported more perfectly.According to the deficiency found in the practical application, we put forward improved HeteroDI system architecture based on XML. In the new architecture, we use adapters to define data sources and channels. It's a flexible way for user to use and add new metadata. Furthermore, we improve the way that the data are circulated and exchanged. In conclusion, the merit and insufficient of HeteroDI are discussed...