Research On The Malicious Workers Detection Technology In Cloud Computing Environments Based On Mapreduce

With the rapid development of the Internet and IT, the social network and e-commerce applications appear as the representative, which have a large user group. In order to deal with the increasing demand for large scale data sets, cloud computing technology comes into being. Cloud computing based on MapReduce, which is a distributed parallel processing model, is capable of efficient parallel processing ultra large scale data sets. Currently, the major cloud service providers make use of the Hadoop to build their own cloud computing platform. Hadoop is the open source implementation of MapReduce. MapReduce computation model is a distributed parallel processing architecture, focusing on the performance of processing the large amount data sets, but its service integrity assurance is relatively week.In the cloud computing environment, one malicious worker’s malicious results will lead to the entire computing task to fail, which make a serious waste of computing and storage resources, and damages the cloud computing customers. So how to efficiently and accurately detect the malicious workers in the cloud computing environment is very important to ensure the integrity of cloud computing service. This paper studies the malicious worker detection technology in the cloud computing, and propose an efficient malicious worker detection framework. The main work is as follows:Firstly, this paper analyze the characteristics of the cloud computing environment. The study of existing cloud service integrity assurance framework based on MapReduce, analysis of its strength and weakness. Then introduce the related technologies in the proposed framework.Secondly, this paper proposes a more efficient malicious worker detection framework. The framework is based on the sub-domain management workers, using different task scheduling strategy for different domains, making more rational and efficient use of computing resources. In order to improve the efficiency of the verification resource, we add hotspots task cache management module, and use probability authentication mechanism.Finally, Systematic theoretical analysis and simulation tests show that the framework presented in this paper can efficiently and accurately detect the non-collusive and collusive malicious workers within the acceptable overhead. It can guarantee cloud computing service integrity well.