The Research Of ETL Key Technology

Implementation of Data Warehouse and Data Integration all need to use ETL tool. Ability of support multiform data conversion source and target, easily software's deployment and usage, various approach of transformation, these are all very important to ETL tool. Existing ETL tool can't satisfy Data Warehouse and Data Integration application completely on the market.In this paper, we research the key technique of ETL tool realization,base on the improvementand research in advance of our ETL tool-SEU_ETL.Aim at the shortage of SEU_ETL, weused CWM to describe the process of ETL.We construct Uniform Data Model to deccribe corresponding logical data access model of physics data source, shield the difference of description in the different data model;construct meta transformation model to describe the logic process of ETL transformer, shield the difference of ETL transformation's realization in specific way carry out;construct meta scheduler/run model, describe ETL running, shield the difference of software deployment. In meta data management, we introduce the concept of dynamic attribute, raised vivid and the design efficiencies of ETL process design, it have better result on old business data processing. We introduce script's transformation in ETL, and expand standard SQL to keep with the request of ETL, carried out to the UDF support, increased the transformer means that can use. We carried out the data cleaning frame with data transformation mutually blend, the defined clean rules can be exexute automatically. We make the certain research to the scheduler of ETL process, carry out extract granularity control.By these improvment, the SEU_ETL has already can satisfy to implement Data Warehouse and Data Integration, carry out the existing transformation type, and for the designer that provided a friendly design- adjust interface.