Data Analysis Framework And Application Of Ethereum Based On MapReduce

The Ethereum blockchain is a distributed data platform maintained by multiple nodes.The data on Ethereum includes a large number of blocks,transactions and runtime information,which has great analytical value.Exploring the Ethereum ”big-data” will mine potential features of Ethereum to better understand Ethereum,will reveal security issues on Ethereum(e.g.Smart Contract Vulnerability)to better defend against attacks.There are many Ethereum data analysis works,which mainly carry out researches from the perspectives of feature analysis and security analysis.However,they face some problems in data collection,data storage,and data processing due to the huge and heterogeneous data.To solve the problems,this thesis introduces the distributed architecture into Ethereum data analysis,deployed as ”Eth Explorer” with Map Reduce framework.It is an efficient Ethereum data analysis framework integrating data collection,data storage and data processing.The data collection layer of the framework supports users to collect the data they need to analyze,and the data processing layer of the framework supports users to conduct efficient data analysis.This thesis proposes 3 approaches to address design issues: i)to facilitate data collection,this thesis designs instrumenting-based method to collect Ethereum data,ii)to address storage management problem,this thesis designs workload balanced multi-nodes storage method,?)to provide scalable and efficient data processing,this thesis designs three general data processing methods,which are Distributed Data Query(DDQ),Distributed Data Processing(DDP)and the Hybrid Processing method(HP).Based on the above framework,this thesis collects all transactions(611,647,042 in total)and smart contracts(21,281,074 in total)generated in the first 10 million blocks(from July 30,2015 to May 4,2020)and analyzes them.The results discovered interesting potential features of Ethereum(e.g.transaction features of ”Ponzi” schemes),detected46 contracts with ”Re-entrancy” vulnerabilities and 9,938 contracts with Gas vulnerabilities.In addition,this thesis also evaluates the performance of the framework from the perspective of data distribution balance,data processing performance and workload balance.The results show that the framework has significant performance improvement and extraordinary workload balance in data distribution and data processing.