Design And Implementation Of Unified Data Management For Paas Cloud

Clouding computing Platform as a Service (PaaS) is a kind of service that provides software deployment, operation and maintenance as service to software developer. There has been a heated discussion about it. Cloud Foundry (CF) is the first open source PaaS platform for the industry, welcomed by the academic and industry.In the PaaS platform, system components and hosted applications generate real time data, including real time performance data and log data. These data present state information of user visit, system operation and exception appearance. In order to monitor PaaS platform efficiently and maintain the operation of service, these data should be managed. So far, the data management of PaaS (e.g., CF) is not perfect. The shortcomings are as follows:important data will be lost when the applications deployed on the platform restart or stop, data sources are disperse, collection methods are various; data format are different, storage method are different; data clear work and historical statistics are not available.According to this situation, after studying the mechanism of CF platform as well as the successful experiences of New Relic in data collection field, we proposed a PaaS data management plan:centralized storage and unified access will be provided for data gathered by agent from CF components. Based on this plan, we carried out the data collection subsystem, data transmission subsystem and data storage subsystem. The main contribution of this thesis includes:(1). researched the operation mechanism of CF components, using New Relic data acquisition experience completed the development of acquisition agent, added data acquisition feature to CF platform;(2). efficient data transmission and real time analysis have been realized with integration of Kafka and Storm, while unified access to NoSQL database HBase has also been provided.(3). we combined all the subsystems into a whole data management system by data flowing through the circle of acquisition, transmission and storage. In the end of the thesis, take the reality into account, we designed several test cases. According to the test results, every subsystem performs well and reaches our goal.