Research On Distributed Data Processing And Aggregation Strategy In Edge Computing Optical Network

With the emergence of a variety of emerging services such as the Internet of Things,virtual reality,and smart cities and their widespread application,it has greatly promoted the evolution of network structures.The traditional cloud computing model can no longer meet the needs of services in terms of bandwidth,delay,and reliability.Because edge computing can sink the computing and storage capabilities of the core network to the edge of the network,it provides a new solution for services with large bandwidth,low latency,and high reliability.However,due to the limited storage and processing capabilities of edge data centers,the collaboration of multiple edge data centers distributed across different geographies will be one of the main working modes in edge computing.This paper abstracts this working mode based on the collaboration across multiple data centers in a distributed manner into distributed data processing and aggregation jobs in edge computing optical networks,where each job usually contains multiple sub-tasks that are processed in parallel.How to jointly optimize the calculation and bandwidth resource allocation and reduce the average task completion time in the collaboration process of edge data centers is an urgent problem.Aiming at this problem,this paper proposes a heuristic algorithm for joint computing,bandwidth resource adjustment,and job scheduling.At the same time,a resource management and control platform for distributed data processing and aggregation is designed and constructed,where the proposed algorithm is experimentally verified.The research work and innovations of this paper are as follows:(1)This paper proposes a heuristic algorithm for joint computing,bandwidth resource adjustment,and job scheduling.Based on the analysis of the problems and challenges faced by distributed data processing and aggregation jobs,and combining the characteristics of edge computing optical networks,this paper separately mathematically models each process of distributed data processing and aggregation jobs in edge computing optical networks.Then,to solve the problem of computing and bandwidth resource allocation during the collaboration of edge data centers,this paper proposes a joint computing and bandwidth resource adjustment strategy.Aiming at the job scheduling problem,this paper proposes a job scheduling strategy based on the sub-task "cut-in".Simulation results show that the algorithm proposed in this paper greatly reduces the average job completion time and improves network resource utilization.(2)Design and implementation of a resource management and control platform for distributed data processing and aggregation.Based on the analysis of the requirements for multi-dimensional heterogeneous resource collaborative processing and edge data center collaboration in edge optical networks,this paper designs a resource management and control platform for distributed data processing and aggregation based on SDN/NFV technology and various open source projects.And combined with the existing programmable optical communication equipment to complete the construction of the experimental platform.In this platform,the flexible management and control of multi-dimensional resources in the edge computing optical network and the distributed data processing and aggregation process in the edge computing optical network are realized.At the same time,the effectiveness and feasibility of the proposed joint computing and bandwidth adjustment strategy are experimentally verified on an experimental platform.