The Design And Implementation Of Big Data PaaS Service Based On Openstack

With the rapid growth of Internet Applications,a large number of data consolidation and aggregation phenomena have caused the demand for data processing applications to explode,accelerating the arrival of the “big data era.” Along with the variety types of data and its expanding scale,data analysis and processing technologies are constantly being improved,and many big data processing softwares are emerged.However,building and deploying a big data cluster is a huge economic and technical expense for Small and Medium Enterprises.The big data processing software cluster is deployed on the cloud platform.The cloud computing technology solves the loose coupling between the upper application and the underlying computing resources from the technical level.As for service,the people who use big data processing software to process data can only focus on the processing of the service while the third-party cloud provider provides administration and maintenance of the big data cluster.According to the urgent need of promoting integration between the cloud computing department and the big data department in New H3 C,the thesis designs and implements the big data PaaS(Platform as a Service)service module based on OpenStack,which can provide users with big data processing software cluster.BDPaaS(Big Data Platform as a Service)is a term typically used to refer to services that offer analysis of large or complex data sets,usually over the Internet,as cloud hosted services.Based on a BDPaaS service about OpenStack cloud operating system and private cloud platform “CloudOS”(H3CloudOS)from New H3 C and aiming at the complex deployment process and difficult management of big data processing software cluster,it implements a big data analysis module for deploying big data processing software cluster on the cloud platform.Firstly,the author analyzes and understands the big data service solutions provided by OpenStack after reading the source code of Sahara in OpenStack.Then,after the overall architecture of CloudOS is introduced,the author learns on the functions that the private cloud platform has realized.Combining the functions of OpenStack and CloudOS,the author performs requirement analysis and general design for BDPaaS.Finally,BDPaaS is implemented according to the design,the relative online testing shows that the design and implementation of this thesis can provide the big data processing software cluster on CloudOS to meet the project requirements.BDPaaS in this thesis mainly realizes the tasks of CURD(Create,Update,Read,Delete)etc in big data processing software cluster on CloudOS.The working focuses of this thesis are about some key points in the design process as following:(1)How to use HTTP protocol to interact between big data PaaS service and components in OpenStack and CloudOS.(2)How to simplify the operation and make it convenient for users to get clusters due to the complexity of the OpenStack big data cluster deployment process.(3)How to embody it in the big data PaaS platform as CloudOS has optimized and changed the user module in OpenStack.(4)How to design and implement a well-defined visual interface with easy operation.