Design And Implementation Of ETL Module For Business Intelligence System

In recent years, grid computing and business intelligence become more and more popular both in research and practice area. Adopted by many companies, grid technology greatly improves the efficiency and utilization of computing resources. And business intelligence technology which aims to turn business data to knowledge helps companies to make more sensible business decisions. Concerning the resource information and job running data of LSF(Load Sharing Facility), a grid cluster job-management system developed by Platform Computing company, an business intelligence system Platform Analytics is proposed to assist company managers to analyze data.Firstly, this paper analyzes the framework of Platform Analytics system, which is oriented to gird environment, deciding the important position of ETL (Extraction-Transformation-Loading) module in business intelligence system. Secondly, through analyzes the requirement of ETL module in PA business system, a configurable ETL module is designed and implemented. ETL module, whose main function is performing data translation, plays a core role in the business intelligence system. According to different functions, ETL module can be devided into ETL engine, Extractor sub-module, Tansformer sub-module, and Loader sub-module. The ETL module designed in this paper can transform raw data to data warehouse according to the configuration files.Finally, according to Platform Analytics application environment, an ETL flow instance is designed to verify the proposed ETL module. The result shows that the ETL module can complete data transformation from database to datamarts; moreover, it has good extensibility and usablity. The proposed ETL module has been applied to the development Platform Analytics, a business intelligence system of Platform Computing company.