Design And Implementation Of Data Integration System Based On Web Services And Quartz

With the era of big data coming, data is becoming increasingly important to companies, it promotes data resources can be shared, exchanged, integrated and reused among enterprises and among the departments in the enterprise. However, due to the differences in business and information construction at different time, these data resources are stored in different network nodes, serving a single business system and gradually forming some information islands. The utilization rate of data is very low and the potential value of data has not been fully played. Under these circumstances of big data age, data integration has become the key role, enterprises can achieve to make the most of data resources through the data extraction, data conversion, data storage and data analysis.The data integration system designed in this paper is aimed to integrate the relevant emergency data, and make it as the theme to collect and integrate the data resources from different data sources, and ultimately provide the data service support for the normal operation of the business system. Based on a great amount of investigation, the system uses Web Services technology combined with the data warehouse to integrate distributed and heterogeneous data resources. The system uses XML to transform data structure to shield data heterogeneity and uses Quartz to schedule web service to regularly update the data in data warehouse.The author participated in the entire life cycle process of the construction of data integration system, from the system requirements analysis、system environment construction and testing、data mapping and database design、code implementation and testing、system operations and performance improvement to the system online. Specific work contents are as follows:(1) Participated in the system requirements analysis, determine the theme、 theme contents and the relationship among these themes and formulated relevant documents.(2) Participated in the construction of the system development environment, through the investigation for the existing data integration solutions, combined with the actual needs, integrated different frameworks and tested the availability of framework.(3) According to determined theme, to design database tables、realize the structure mapping between the database table and the XML Schema and generate XML Schema definition files.(4) Participated in a lot of development and testing works of Web service client application.(5) After the system was on-line, gradually found that the performance of the system got progressively worse, so the author used JAXB to parse XML to optimize the system performance and established the partition to the database tables to promote the inquiry speed of the database, thus optimized the database performance.Data integration system has been running on the line, the integration of data from more than 30 different data sources, all of these data were subject to the emergency theme, these diversified and reliable data supported for the effective operation and the assistant decision of the emergency management system.