Edge Distributed Based On Cloud-Edge Collaboration Design And Implementation Of Storage System

With the vigorous development of edge computing,more and more computing tasks will be sunk to edge nodes,and these computing tasks will collect,process and store massive data at the edge.As the amount of data continues to grow,the limited storage capacity at the edge will no longer be sufficient to support the amount of data generated by edge computing.Most of the storage scenarios for traditional distributed storage systems are large storage rooms,and for the sake of versatility,they often provide file,block,and object storage functions at the same time,which will make the storage system too bloated.Excessive consumption of edge storage and computing resources,thus affecting the execution efficiency of edge computing tasks.Therefore,aiming at the problems of limited edge storage resources,complex storage environment,and inaccessible network,the thesis designs and implements a lightweight,easily scalable,and highly available distributed storage system for edge scenarios based on Kubernetes.The main research contents of the thesis are as follows:1)Considering that the traditional distributed storage system is not closely related to Kubernetes and cannot give full play to the native capabilities of Kubernetes,this system strips some functions of the storage system and implements it with the help of the native functions of Kubernetes,thereby simplifying the overall architecture of the traditional distributed storage system.Write a cloud controller group based on Kubernetes to manage and control edge storage resources.The controller group collects edge storage resource usage in real time,formulates efficient storage plans for storage tasks,and distributes storage tasks to appropriate edge storage clusters.2)The edge network environment is complex.This system designs and implements a proxy service grid framework capable of cross-network communication.The proxy service grid allocates network segment resources through the cloud controller group,and connects the cross-network nodes in the form of tunnels at the edge.Storage nodes,to achieve horizontal expansion of side storage nodes.In order to deal with excessive redundant service grid resource occupation at the edge,the thesis proposes a multi-agent cluster partitioning strategy to divide the redundant grid at the edge,thereby reducing the resource consumption of the edge.3)Aiming at the complex and changeable storage environment of the edge,a multidimensional feature storage resource scheduling algorithm is designed and implemented.The edge storage master service collects and updates the storage feature information of each storage node in real time,from the storage node CPU,memory,storage space.network delay,neighbor storage resource topology map and other multi-dimensional calculations to determine the scheduling and allocation of storage resources.In the simulated edge storage environment,the system has performed functional tests and performance tests respectively.The test results show that the functions of this system can operate normally in edge scenarios.In addition,by comparing with traditional distributed storage systems,this system has Slightly better performance.