Verifiable Computing Based On MapReduce In Cloud Computing Service

Cloud computing providers computing resources as a service,which greatly improves the efficiency of big data analysis and processing.With the popularity of Hadoop,Map Reduce computing framework has become one of the most fundamental methods for big data processing.In the cloud computing service when the Map Reduce program is running,the wrong configuration of the cloud computing platform,data corruption in storage or transmission,hardware problems,malicious operations and so on will lead to the wrong calculation results.Whether computing in cloud services can be verified becomes the core index to measure cloud computing service's quality.This paper focuses on the verifiable computing based on the Map Reduce in cloud computing services,and implements a log-based auditing runtime integrity verification system to achieve the purpose of verifying program execution integrity and completeness of results.The main contibution of this work are as follows.In order to solve the problem of how to verify the reliability of computing in the cloud service,we propose an interactive verification method to verify the reliability of the verification information generated by the untrusted cloud,and a basic solution of the runtime verification of the program based on Map Reduce in cloud environment.The basic solution mainly designs the program checkpoint setting and insertion method and the log-based data integrity verification method to achieve verification of data input and function execution,which can not only detect the control flow tampering,but also detect the effect of data tampering.The security anylysis of this solution proves that the program performs complete and complete results when the integrity verification result is passed,and realizes the credible verification of the whole life cycle of the program execution.In the basic solution of runtime integrity verification,the trusted verification of a program's whole life cycle means the transmission and analysis of massive logs.In order to improve detection efficiency,this paper proposes efficient verification methods for execution verification and input verification respectively.This paper proposes a log probabilistic verification method based on merkel tree,which ensures that the trusted end can retrieve part of the log safely according to its needs,so as to achieve the goal of probabilistic verification of the log on the trusted end.For input validation,a probabilistic input datachecking method based on counting brun filter CBF is proposed to improve the efficiency of input validation.The security of the two methods is analyzed theoretically.The results show that when the input data set of the integrity verification scheme using these two efficient verification methods is 1TB and only 1% of the log in the basic scheme is used,only 1.4% CBF overflow rate and 16.6% miss rate are introduced in order to detect the pollution data with more than 3 records modified.Based on the above proposed techniques and methods,a runtime integrity verification prototype system based on log audit is designed and implemented.The suitability and performance were tested using a variety of Map Reduce applications.Experimental results show that the runtime integrity verification system is suitable for runtime integrity verification of all Map Reduce applications in the experiment.In terms of overhead,in Map Reduce's word counting application,the correct parameter setting introduces only 45% of the extra execution time on the trusted node and 28.91% of the extra execution time on the public cloud.The time overhead is less than the Pantry system based on proof verification.