Design And Implementation Of An Efficient Blockchain System For Massive Data Storage

With the popularity of Internet of Things devices and the growing demand for artificial intelligence applications,these applications generate a large amount of Internet of Things data.In order to achieve the beautiful vision of more convenient communication between things and people,it is necessary to consider the transmission and sharing of Internet of Things data.How to extract value from massive data while protecting user privacy and encouraging more users to participate has become a question that must be considered.Therefore,the advantages of blockchain technology and machine learning make it an indispensable tool for intelligent development in fields such as the Internet of Things and big data.However,traditional machine learning using centralized training mode cannot solve the "data islands" and data privacy issues.Federated Learning is a machine learning technology that can train models across multiple devices or nodes without transferring data from these devices to a central server.The use of federated learning can protect data privacy and security,reduce data transmission bandwidth and accelerate the training speed of the overall model.Although federated learning provides better data privacy than centralized learning,it still poses security issues,such as the possibility that certain parameters may be tampered with during model transmission,resulting in reduced model accuracy,and the need for continuous communication between the device and the central aggregation server resulting in higher latency.Blockchain provides a decentralized and tamper-proof ledger that securely stores model updates and facilitates communication between devices and central servers.As a point-to-point distributed file system,IPFS can store large files in a distributed network and has good storage scalability.Using blockchains and IPFS in federated learning can achieve data sharing and collaboration among multiple parties,while protecting data privacy and security.This thesis first introduces IPFS technology to optimize the storage performance of the current blockchain,and then combines it with federated learning technology.The main research content of this thesis is as follows:(1)Aiming at the problems faced by the actual deployment of blockchain systems,such as the difficulty in estimating performance and determining the performance of required hardware facilities,this thesis establishes an accurate expression model for blockchain network throughput based on model and data fusion under different network structures.By analyzing the transaction process of the PBFT(Practical Byzantine Fault Tolerant Algorithm)blockchain system,taking into account the network topology,consensus algorithms used,and communication methods of nodes,and based on a large amount of actual data,a transaction per second(TPS)accurate expression model for blockchain transactions based on the Fisco-Bcos platform has been established.The actual blockchain system test results show the TPS prediction model established in this thesis can maintain high prediction accuracy under different network structures.(2)Blockchain improves the security of the system by storing a copy redundantly on each node,but it cannot be ignored that this method also significantly reduces the effectiveness and scalability of the system,making it difficult to implement blockchain in some scenarios that require lightweight storage applications.This thesis proposes a blockchain and IPFS based blockchain storage model to address the low scalability of blockchain storage.This model alleviates the local storage pressure on blockchain nodes by building an IPFS file system off the chain,placing model data with a large amount of data on the IPFS file system,and storing only the address of the data on the block.The actual test results of the system show that the model can effectively reduce the local storage load of blockchain nodes,and can fully utilize the remaining storage space in the network.(3)A federated learning system based on blockchain and IPFS is designed and implemented.The system is mainly aimed at operators,enterprises,and consumers,and is suitable for business identification of traffic data.Through federated learning,distributed learning of corresponding APP traffic data is conducted.Each node generates a model locally,and places the model update data with a large amount of data into the IPFS file system.Only the hash address of the model is stored on the block.The actual test results show that the system can effectively alleviate the problem of excessive local storage pressure on blockchain systems in federated learning scenarios.