Research And Design Of A Universal ETL Tool

The data warehouse can effectively put a lot of distributed and heterogeneous data integration to the unified data-environment in order to provide decision-making data access. Improving enterprise resource utilization data capacity plays a more and more important role in the modern enterprise information construction. A high quality ETL process is the key point to construct data warehouse. The heterogeneous data is extracted and conversed and loaded into the data warehouse by utilizing the ETL tools based on the application of decision analysis to provide high quality data.The ETL tool is inherently a data conversion tool. The traditional ETL tool is normally loaded into the memory data by converting or quality detection while conversion efficiency is relatively low. In addition, the traditional ETL are oriented in a particular field. It is better to support the specific transaction in this filed than in other fields with new business logic which is could be very difficult to deal with.According to the traditional ETL tool defect, this paper has been deeply analyzed and discussed to concentrate on the study of several key issues to offer a general design and system architecture of ETL tool. It supports a variety of heterogeneous data platform. To provide a large number of fine-grained conversion component in conversion step by the way of component combination to complete the complex affairs in order to support multiple business demands. The tool from the heterogeneous data source to obtain metadata and loaded into the dedicated ETL server. The design of before and after mapping in ETL server and entire process (metadata of extraction rules) are saved in XML. In the operational phase, the corresponding SQL script is generated through the process of saved XML analysis and handed over to the ETL server to complete the entire conversion work. In the end it is loaded into the target database.Finally, the realization of such a generic ETL tool is applied to some specific business situations. It is a flexible and convenient tool and running smoothly for developers to complete the ETL process of data warehouse effectively.