Research And Implementation Of OLAP System Based On Cloud Computing Platform

The advent of cloud computing addresses perfectly the problem of processing mass data. To further promote the development of our marine information technology, cloud computing can be used to construct the system of the marine environment information service application framework. It can also improve reusability and sharing of the information of marine resources and extendibility of application system. In this paper, an OLAP system based on cloud computing platforms has been created referring to national welfare projects of ocean. The system provides its users multidimensional views, through which users can, from diverse angles and different levels, observe and study the data and comprehend them in depth.Existing OLAP system has a problem of low efficiency in data processing and absence of data information when dealing with large-scale data set. These problems have become the biggest bottlenecks for current OLAP system. OLAP system based on cloud platform can not only store massive amounts of data and calculate parallel computing data cube, but improve the computational efficiency and reduce system response time. The OLAP system, built on cloud computing platforms, has three layers-storage layer, engines of OLAP and application layer. The layer of accumulation uses the data warehouse, Hive, which has its own SQL-like language, HiveQL. The System has used JAVA language to implement a Hive dialect class, HiveDialect, on the basis of common interface method provided by Mondrian, the engine of OLAP. The HiveDialect can transform SQL produced by Mondrian to HiveQL, realizing the engine’s access to data of the accumulation layer. The engine of OLAP, by using open source project, Mondrian, has made the mapping of physical models and multidimensional models, created the cube and analysed MDX. The application layer adopts a custom tag library, JPivot. The layer uses the tag library, JSP, to provide buttons related to the operation of OLAP and presentation of data, including display of images and forms and connection to underlying data models. Besides, the application layer supports access to data sources in the way of JDBC.According to the needs of the project, OLAP system should provide its users a classification mining algorithm library based upon cloud computing platforms and assist them with further data mining and analysis. The existing distributed machine learning algorithm library, Mahout, offers some classic classification algorithm, such as Bayes and decision trees. On the contrary, OS-ELM, known for its high learning speed and good generalization performance, is not realized on cloud computing platforms. In this dissertation, therefore, parallel processing of OS-ELM has been carried out and a Map-Reduce-based OS-ELM, MOS-ELM, hase been designed by the usage of the frame of Map-Reduce. Simulation experiments indicate that MOS-ELM is feasible. Compared to centralized OS-ELM, MOS-ELM has advantages of simple models, easiness to achieve, good augmentability and parallelism.