Research On Performance Optimization And Visualization Tool Of Hadoop

MapReduce is a sort of programming model to process the massive data. Many important problems in cloud computing such as search engine service, large scientific computing tasks, the massive data mining could be resolved by the MapReduce, thus it plays a vital role in cloud computing. With the MapReduce applied in more various fields, performance problems of MapReduce have also been attracted the scholars’ attention. As the development of visualization technology, the visualization of performance optimization process and results and the visualization of the learning process have become a research hotspot in recent years. Hadoop, as the most popular Java implementation of Google’s MapReduce programming model, is certainly to be the most important platform to study. The performance optimization and visualization tools development in this paper are based on this platform.Hadoop plays a crucial role in the operation of the scheduling process. A sound scheduling algorithm can make sequence execution and resource allocation more scientific and efficient, at the same time can improve Hadoop performance and resource usage. First, performance optimizatio. The study for FIFO (First In First Out) in this paper has found that it does not achieve the data locality-the data locality is not so sound even in a small scale of operation. Therefore, FIFO is modified to improve data locality. Second, Hadoop cloud computing platform. By contrast and analysis for the improved and the original FIFO, the results show that the improved one could be better in data locality of operation, time efficiency of the data transmission of local tasks. Thereby, the total operation time is shorter, the system throughput is improved.As for visualization, the visualization tools are developed in this paper, which are applied in performance visualization and learning visualization. (1) Performance visualization. As the performance optimization is combined with the Hadoop visual management platform, a new sort of visualization tools is developed on the basis of performance optimization. It could let users to operate data visually, and make users choose different display forms and different display platforms. At the same time, in order to be compatible with more performance optimization methods, the interface is exposed to support the function expansion. (2) Learning visualization. A visual learning tool—cloud computing virtual experiment teaching system is developed for cloud computing learners in this paper. The system contains 12 experiment series of cloud computing, and the most important parts are the basic knowledge and the theory applied in the practice. All of these experiments are to be demonstrated by vitalization. In advantage of interaction, immersion and imagination of the experiment, the design could make up for the disadvantage of classroom teaching and experiment teaching, break the limitation of space and time and help students understand the basic composition and technical principle of the cloud computing and Hardtop. Virtual experiment is combined with Java technology and Flash technology and with the video in assistance, so that students could fully understand all the details of the experiments. More of all, the experiment simulates some common errors and gives the processing methods, so that the students could recognize these errors in the experiment and avoid damage for experimental equipment in the actual operations.