Design And Application Of Data Processing Platform Adapting To Changes In Demand

With the development of computer technology and penetrate deeply application of the Internet, enterprises faces the tendency of the explosive growth of data. How to mine useful information from these massive, diverse, heterogeneous and dynamic data by the way of accurate, efficient and low cost could help enterprises to make decisions, which has become an important goal for many enterprises with next generation information system. The traditional development model must determine the functions of the application before development. So it is difficult to adapt to the rapid growth of base data and the change of application environment, which is very easy to cause the long development cycle and short service life for application system, and can cause huge man power and material waste. Therefore the research of how to make the data processing system adapt to changes in demand has practical value.This thesis on the base of the foundation of the theory and technology of application system that can adapt to changes in demand, a scheme has been proposed in this thesis:Based on general platform for dynamic data processing, the function of custom data processing process is provided, while application system through a process execution engine to execute process or combine process, that is to enhance the ability for applications to adapt to the changes of data and demand. The main contributions of this thesis are as follows:Firstly, this thesis research on the cross platform exchange statistics and mining model language of PMML. Based on the study of the ministry of open source PMML execution engine, the PMML execution engine is designed in this thesis.Secondly, the design and implementation of a set of support data preprocessing, data integration and data mining visualization data processing platform is completed. This platform uses PMML language as the basic data processing description language. The independent design PMML execution engine is used as the customization module core, which allows direct and interactive drawing mode, the visualization of process design and process execution, the visualization of test execution results and other functions.The main function of the platform has been basically completed, which can operate normally. The platform is applied with some massive bank customer transaction credit analysis of financial data, to deal with some variable demand methods through custom data processing. The adaptability of system, especially those for high level application, is enhanced. The effectiveness of this scheme is verified.