Research On Verifiable Computation Based On MapReduce In Cloud Computing

With the development and integration of global information industry,network resources and data size are growing quickly.Traditional data analysis techniques can no longer meet the requirements of the current age.Cloud computing,which provides computing resources as a service on demand to the majority of internet users,emerges as the times require.It integrates the idle resources in the server cluster,and realizes automatic management through specialized software,which can provide an effective solution for mass data parallel computing and distributed storage.MapReduce has become the mainstream processing model in cloud computing with its advantages of easy to use,high scalability and low cost.Most of the existing open data processing services are mainly based on MapReduce distributed parallel computing.However,the distribution of workers providing computing services for the MapReduce model is extremely extensive and we can not ensure that all workers are safe and credible.Furthermore,the erroneous results produced by a single worker may lead to failure of the entire task,it is necessary to verify the results of the workers.The existing result verification scheme of MapReduce mainly uses the multi-copy technique,which replicates the task multiple times and assigns them to different workers,and verifies the correctness by comparing the returned results.But this method will be ineffective if the workers collude with each other and return the same erroneous results.We further study the existing verification schemes,and propose an effectively verifiable computation scheme of MapReduce based on attestation graph.The main contributions of this thesis are as follows:(1)We deeply study the existing verifiable computation scheme based on MapReduce,and we figure out the deficiency of adversary model assumed by existing schemes and consider stronger adversary model where the malicious adversaries can selectively output the computation result when processing identical task with other unfamiliar malicious workers.On this basis,we propose a verifiable computation scheme based on MapReduce in cloud computing.The scheme adopt consistency clique analysis algorithm based on attestation graph,which can find the malicious workers quickly and accurately by using the consistency relationship among workers.In order to improve the detection efficiency,we calculate the workers' trusted value through their historical task information to guide the follow-up detection work.Furthermore,we use the method of probability derivation to analyze the proposed scheme in detail.The results of the theoretical deduction show that our scheme can detect the inconsistent relationship among workers quickly and improve the detection efficiency.(2)The simulation environment is built by setting different parameters in the Linux system,and we conduct an experimental evaluation of the detection algorithm in our scheme.Furthermore,we compare with the related scheme in the same environment.The simulation results indicate that our scheme can detect the collusive and non-collusive workers quickly and accurately with proper computation overhead,and improve the result accuracy of MapReduce.