Research And Implementation Of Massive Data Analysis System Based On Cloud Computing

With the development of information technology, data analysis is playing an increasing rolein business decision and product design. Based on analysis of user behavior data, productdesigners can design functions with an enhanced user experience and can improve functionsof other parts. Decision makers of a company can use the result of data analysis to decideproduct orientation and the company’s developing direction. However, the development ofthe company, the increase of products and users and other factors lead to a huge increase ofthe total quantity of data. Facing the challenge of data saving and handling capacity,traditional data analysis system cannot able to accomplish its mission.This paper mainly discusses developing a new mass data analysis system in face of theabove challenges, based on cloud computing technology, cloud service provided by Amazonand HBase. Through the research of the EC2, S3and EMR service provided by Amazon anda detailed analysis of the client, reception server and data handling server of the data analysissystem, this paper tries to create a preliminary model of the system. Then it tries to finisheach model and whole system according to actual demand. Finally, a system test andperformance test proves that the system can solve the problems faced by traditional dataanalysis system or not.The research runs through the whole process of data analytics system design andimplementation. Firstly, present the problem domain and requirements. We throughinvestigate the traditional data analysis system and analyses the problems existing in thissystem, then raise the developing requirements of the massive data analysis system based oncloud computing, and clear the key points of the new system implementation are receivingserver and data processing design. Secondly, design and realize each module of the massivedata analysis system. Finally, execute the system test and performance test on this system, thetest results proved that the new data analysis system is stable and can supply therequirements.Currently the cloud computing and data analytics system are in the boomingdevelopment period. This research runs through research the cloud computing and Amazon could Web services, and used them in enterprise data analysis system implementation. Wefinished the data analysis system based on could computing, so that the problems which existin traditional data analysis system have been resolved, and aroused data analyzing andhandling capacity of the system, saved the cost, and cut down development cycle.After using cloud computing technology, data analysis system is more flexible in dataprocessing and analysis; it can increase or reduce the computing resource according the actualrequirement. Also the spending of equipment maintenance and data backup will be saved.Both from the data processing system performance and cost of the project, using cloudcomputing technology will be the best solution. In this study, the actual project successfullyconfirmed and embodies the advantages of cloud computing technology. In the research andapplication of cloud computing technology, this paper has some certain reference value.