The Cloud Computing Based On Hadoop Platform And Log Analysis

Cloud computing is a kind of new typed calculation model, the distribution ofcomputing tasks in a pool of computer resources, enabling users to access tocomputing power, the storage space and information services when they need. Tocompare with the traditional data processing mode, cloud computing technology caneffectively solve the mass data processing faces performance bottlenecks and toimprove the reliability of the data processing and expansibility, improve the ability tohandle data at the same time reduces the computational of hardware equipmentrequirements. In this paper, we study on the cloud computing concept, type, and keytechnology.Hadoop is an open source distributed computing platform, it designed fordealing with large data and distributed computing and design, which is the maincomputing clouds can choose one of the way. Hadoop platform have efficient,reliable, expandability etc, it’s two main component is Hadoop distributed file systemHDFS and parallel processing MapReduce model. In this paper, the HDFS aspects:design premise and purpose, system structure, security measures and improve thereliability and performance measures MapReduce aspects: logical model,programming model and realization mechanism and implement the process of acareful analysis and research.On the analysis of the original mass data processing system after a combinedwith cloud computing and the advantage of Hadoop, established a new dataprocessing model, according to the model built the system platform, and use the Weblog as source data on the performance analysis of the platform. Through the contrast,summarizes the use of cloud computing technology, makes the log analysis processin consumption time shortens greatly, and with the increase of the quantity of data,Hadoop platform processing power and data storage capacity to adapt to the data inthe quantity change, on the embodies the cloud computing technology in dealing with large data calculation power, storage space, according to the needs of improvingadvantage. Based on the cloud computing environment Hadoop platform in dealingwith large data in the data processing method to solve traditional computing powerand storage capacity performance bottleneck problems, and good scalability makesthis ability can be flexible use.