Research And Implementation Of Decentralized Storage System Oriented To Dynamic Network

With the continuous development of chip manufacturing processes and Internet of Things technology and the popularity of smart devices such as mobile phones and tablets,more and more mobile computing devices have begun to appear in various application scenarios,and their performance has been continuously improved.However,due to the heterogeneity of the equipment and the dispersion of geographical location,a large number of computing resources have been idle for a long time,resulting in low resource utilization.As a result,the concept of decentralized computing began to emerge,and its core idea was to use dynamic networking to build a resource-centric architecture that made full use of the decentralized computing resources in the network.In decentralized computing,nodes divide the tasks that need to be performed into fine-grained atomic tasks,and then assign atomic tasks to idle nodes in the network through various scheduling algorithms,reducing the completion time of the overall task in a multi-node collaborative computing manner.Because tasks cannot be separated from the dependence on data during execution,in order to ensure efficient and reliable execution of tasks,a set of data storage schemes suitable for decentralized computing is required.Most of the existing storage solutions are based on reliable networks in the production environment,and are not suitable for the dynamic network environment in decentralized computing scenarios,so this paper studies and implements a decentralized storage system for the dynamic network environment,providing underlying data support for functional modules such as upper-level task scheduling and execution,avoiding data loss problems caused by unexpected downtime and resource exhaustion,and improving data acquisition efficiency.The storage objects covered in this article mainly contain unstructured data and task-related relational data.For unstructured data such as pictures and documents,based on the locality characteristics of data acquisition in decentralized computing scenarios,this paper first proposes a redundant placement scheme of local data based on node hops,which ensures the reliability of the data,and at the same time ensures that nearby nodes can access one of the replica data with a short route hop number.After considering that the farther the nodes are,the greater the data transmission overhead,this article divides the data acquisition into two successive steps: local and global,the data request node takes precedence in the local scope through a cache list,and if the local lookup fails,it is found in the global scope through the lookup strategy based on the Kademlia protocol.For task-related relational data,this article refers to Raft to implement a local data storage scheme based on SQL replication.This scheme uses the task scheduling node as the source node to synchronize the task data required by the scheduling node to the nodes within the specified range to ensure the reliability of the data.At the same time,aiming at the problem that the scheduling node may fail,an election strategy for the successor node is proposed,so that the node with higher data synchronization efficiency has a higher election success rate.Based on the above technical research,this paper designs and implements a decentralized storage system containing unstructured data and relational data in detail.Then,using the Raspberry Pi,a verification environment for the decentralized computing architecture was built and tested in detail.The test results show that the storage system implemented in this paper can reliably store different types of data in the decentralized computing dynamic network environment,tolerate dynamic characteristics such as node failure,joining,and leaving,and meet the requirements of decentralization and meet the characteristics of data access locality.