Font Size: a A A

Research On Distributed Storage Technology Based On Edge Node Cluster

Posted on:2021-11-27Degree:MasterType:Thesis
Country:ChinaCandidate:F X ZhouFull Text:PDF
GTID:2518306503999129Subject:Software engineering
Abstract/Summary:PDF Full Text Request
With the advent of the 5G era and the rapid development of the Internet of Things,the IoT smart devices are becoming more and more popularize,and the number of them has increased rapidly.Various types of monitors,sensors,and mobile smartphones used by people generate a lot of data every moment.The improvement of hardwares' performance makes the edge of the Internet such as macro base stations,main switches,gateway servers,edge servers and even routers have some computing and storage resources.Therefore,the concept of Edge Computing was proposed to transfer the realtime calculation and preprocessing of data from the cloud to the edge,which not only saves the bandwidth,but also reduces the response delay.Due to the main application scenarios for edge computing are widerange nowadays,such as smart cities,connected vehicles,and urban computing.However,in some small-scale and local area scenarios,such as a college campus,a parking lot,an industrial park,or even an office building.And these small-scale scenarios also need the computing and storage resources of edge nodes to reduce latency and save bandwidth.Considering these scenarios,which have the characteristics of small volume of data and limited funds.Therefore,in this paper,we will study a decentralized edge storage architecture uising edge node clusters to solve storage problems in small-scale application scenarios.This paper abstracted storage as a service(Storage as a Service)and proposed a decentralized edge storage service called DECS.This paper designed the decentralized architecture of edge storage firstly.And then studied the data storage strategy in DECS.Before reading and writing data,end devices will weigh through the trade-off module in DECS and pick the optimal target edge node to read or write.This stragegy makes users writing data to the edge nodes that are most likely to be accessed,in this way,end devices can have lower latency when reading data.This paper introduced the data rebalancing strategy in DECS subsequently.We designed a prediction module for DECS,and this module can periodically use the historical access records of data to build a machine learning model,priori to predict the access number of data in the future.Build forwarding rules for data,or replicate,migrate and distribute popular data.The construction of forwarding rules optimized the write direction.Or else,content distribution and replica strategy made popular data to be accessed more quickly.Both of them reduced the read latency.At the same time,the space collection strategy in DECS can recover redundant data in the edge node,and saving storage resources in the edge cluster.This paper described the specific modules of DECS,including the metadata module,data storage module,data processing module,prediction module,trade-off module,and space collection module.In addition,this paper introduced the algorithms and formulas used in detail.Later,this paper used Spring Cloud microservice to encode and implement DECS.About the implementation details of DECS,class entity design,process interaction and other aspects,this paper has introduced in detail.Finally,this paper used the docker container to deploy DECS into a decentralized edge node cluster.And used two real small-scale application scenarios to experiment and verify.The experiments in this paper verified the necessity of the filter in prediction module of DECS and the accuracy of the prediction model.The effectiveness of each algorithm in the data storage module and the tradeoff module was measured by read access latency.And the data rebalancing strategy in DECS was measured by the homogeneity of the edge cluster resource distribution.Comparison with other state-of-art edge storage service model proved that the decentralized edge storage service DECS proposed in this paper is more suitable for small-scale edge clusters with limited resources and uneven resource distribution.And DECS serves the purpose of reducing response latency,saving network bandwidth and resources of edge cluster.
Keywords/Search Tags:distributed storage, edge computing, storage as a service, decentralized, resource management, edge-collaboration, small-scale
PDF Full Text Request
Related items