Support Etl Evolution Of Data Management And Applications

More and more enterprises consider data warehouse as data integrations platform,which provides a single version of truth about data to analytical users to make decisions.The extraction,transformation and loading(ETL)process ,as an important part of datawarehouse architecture,integrates heterogeneous sources data , improves data quantity,andefficiency delivers valued data to end users. Due to environment complexity, thedevelopment of ETL system is not a step but an increment-iterative of lasting process. Inthe increment -iterative of lasting process, ETL evolution must keep the exiting ETLsystem running, but also adapt to reflect changes. The performances bottleneck, descriptiveinformation availability about data and adaptation to changes are unavoidable to ETLevolution. System understanding is after the collection, the analysis and abstractly obtainsthe system information the process, is the precondition that solves the system evolution tomeet the question. The question arise, then, how gain the ETL system information to beable to be helpful to solves the performance bottleneck, the data information usability andthe adaptation schema change question which the ETL evolution faces.The metadata entrusting with the system content significance the descriptioninformation, becomes the key to system understanding. In order to solve the above problem,how to manage and apply metadata to supports the ETL evolution turns the key point thatstudies for this paper. Based on analysis of ETL metadata, the paper presented Metadatamanagement architecture as solution of ETL metadata management and application.Employed UML modeling language, Power designer tool and structure model, ETLmetamodel was completed. To make accession to metadata convenience, the metadatarepository has been built on SQL Server2000 with the orient-relationship mappingtechnology. Through inquiry tool of SQL Server 2000 or other custom-made the applicationprogram, user uses the SQL to directly access the ETL metadata repository. The metadataaccessed from repository can be used for performances bottleneck diagnosis, metadatabrowse,data linage ,data quality and impact analysis。These appliances can help ETL solvethe problems, which met in the process of ETL evolution, and consequently support forETL evolution.