Research On The Novel Big Data Processing Platform In Cloud Computing Environment

The fluidity and richness of human activities,with the production of multimedia and other digital content,"Big Data Revolution" is taking place.Big Data is known as data-intensive technology,it is becoming a trend in the development of new technologies for scientific,industrial and commercial fields.Big Data platform as the infrastructure of Big Data for data storage,management,query,and analysis phase to provide resources and services,which is an integral part of a large data processing.Big Data has a variety of data sources and field of use,on the one hand,the arrival of the era of Big Data,for social change,enterprise development,scientific discoveries brought new opportunities.On the other hand,Big Data because of its high technology threshold brought challenges and difficulties for enterprise applications and personnel training of which the main challenge is cluster planning,installation,deployment and other operational needs,and Big Data computing,mining and other processing requirements.Meanwhile,the research of big data platform,in academia and industry has the following characteristics and trends:(1)Big Data Processing Platform(BDPP)built on cloud is the development trend;(2)BDPP makes Hadoop Ecosystem as the underlying basic services;(3)BDPP will be more streamlined,the threshold will be lowered to build a platform;(4)more upper application will be integrated into the BDPP.In order to meet the challenges of Big Data platform brought to production,combined with the characteristics and trends of research platforms,this paper will make New Big Data Processing Platform under Cloud Computing as the research object,from the overall architecture to base resource platform,application services and data stream three levels of study and demonstrate the availability and stability of the platform by way of instance.Among them,the base resources layer as a platform to provide a steady stream of cloud resources(cloud instance,cloud hard drive included),application service layer is responsible for platform applications(big data tools)deployment and maintenance,data stream layer to the design and definition of processes,the entire platform services for the entire life-cycle of Big Data.