MapReduce Computing Model Designed To Performance Optimization And Applications

Cloud computing is not a new technology,it is in the parallel computing,virtualization,grid computing technology on the evolution and development of more mature technology level up to now.It is now beyond dispute to become one of the development trend of computer science.Cloud computing itself is a kind of thinking mode.giving full play to its own advantages,it not only needs hardware facilities,also haves a programming model to support and implement the idea of cloud computing.And MapReduce programming model can realize the thinking of cloud computing,used on different cluster can be distributed to perform user submit assignments,and its performance and fault tolerance has become a research hotspot.In order to improve the execution efficiency of MapReduce programming model,the Map output transmission optimization is proposed.First of all,set up a combined file size threshold,through the Map tasks in the node according to the Map task end time simultaneously or successively,the output of its multiple Map the task will be merged output file,but the combined file size must not exceed the threshold.So that we can effectively shorten the transmission time to Reduce phase,thus improve the execution efficiency of the system.In this paper,using the original graphs fault-tolerant mechanism is optimized.By introducing a spare monitor instant messaging system,it is through the same shelf free TaskTracker node to check other TaskTracker node of the current situation.If a node was found to be a failure,the node failure message will be sent immediately to the JobTracker node,without waiting for a heartbeat cycle,other information through the heart still communication mechanism to push the JobTracker node.It also can shorten the shorten the time to find and correct the failure failure node.This paper Improvement of MapReduce used in innovation knowledge cloud platform,in cluster designed and implemented innovation knowledge cloud instance classification management system.And proved through the experiment,the efficiency of case classification algorithm based on the improved MapReduce model has been effectively improved,so as to improve the classification management system of innovation knowledge cloud platform instance execution efficiency.