Research And Implementation On Message-based Event Transmission & Automatically Processing Mechanism In Real-time Data Warehouse

With the widely using of business intelligence(BI), data mining(DM) and OLAP in modern enterprise, as the foundation of these technologies, data warehouse gets more attentions. It provides subject-oriented, integrated, nonvolatile and changes with time data for data analysis of enterprise.Traditional data warehouse effectively supports decision-making on the strategic, but with the development of computer technology and the needs of market competition, many enterprises have not been satisfied with the cycling updating data in traditional data warehouse, which can not supply the newest data for timely analysis results, so as to missing the best time for decision-making and losing of market competitiveness. Therefore, based on the traditional data warehouse applications, many enterprises also need provide tactical guidance to the daily operation of the business unit. So, real-time data warehouse (RTDW) is occurred to supply freshness data for analyzer.RTDW is a new research direction in the technology and application research of data warehouse, aiming at designing a solution which timely provides the latest data and effectively fuse with historical data for tactical and strategic decision-making. However, the implementation of RTDW has many problems that need to be resolved, particularly in the transformation based on the tradition data warehouse.The thesis firstly introduces the basic concepts and features of RTDW, detailed analyzes the problems of implementation in real-time data warehouse. Then, it designs a viable real-time data warehouse framework scheme from real-time user-oriented concept, which uses ODS preserving real-time data, cycling loaded real-time data into the data warehouse. Meeting the need of accessing both real-time and historical data, application interface uses real-time data fusion technology to provide data and solve conflicts of real-time updates with traditional data warehouse physical store.Secondly, after capturing the real-time operational data in the RTDW, real-time or timely incident processing and automation decision-making is essential to RTDW. Against the problem of resources competition, the thesis puts forward an asynchronous event-driven automation scheme which is reliable, scalable, and load balancing, by applying a message queue and the improved active rules techniques. Furthermore, it also presents a practice reference scheme which uses advanced queue middleware and J2EE technique.Finally, this thesis gives a realization of the solution and performance tests. The experimental results show that the schema proposed in this thesis is feasible. The jobs in the paper have some reference values for enterprises to realize real-time decision-making. 东北大学硕士学位论文AbstractKey words:real-time data warehouse；event message；active rule；architecture；message queue...