Machine Learning Based Generalized Resource Allocation Scheme In Metro Optical Networks

With the advent of the 5G era,emerging businesses such as virtual reality,the Internet of Things,and 4K high-definition video streaming have developed rapidly.These high-bandwidth and low-latency services have brought new challenges to communication networks.In order to meet the growing demand for traffic transmission,optical network gradually sinks from the core bearer network to the metropolitan area network.At the same time,a large number of intelligent wearable devices,Internet of Things devices,Internet of Vehicles devices,etc.have emerged,giving birth to the edge computing service.The user equipment uploads the tasks that require a large amount of calculation to an edge server,and the edge server completes the calculation of the tasks and transmits the results back to the user equipment,so as to reduce the processing time of the task and the energy consumption of the user equipment.Therefore,with the development trend of cloud network convergence,in addition to fiber links and optical nodes,there are also some edge computing servers distributed in metro networks,which are used to support edge computing services.The business of 5G era demands increasingly high bandwidth and low latency,so the complexity of metro optical networks put forward higher requirements for control schemes.The performance of control schemes directly affects the capacity and quality of service in metro networks.This paper focuses on the joint allocation of communication resources and computing resources in the cloud network integration scenario,and the cooperation scheme between edge clouds when processing edge computing tasks.The goal of this paper is to improve the throughput of metro optical networks for edge computing tasks without increasing network communication resources and computing resources.The work of this paper mainly includes the following three aspects:The cloud network convergence scenario and edge computing services are analyzed,including spectrum resource allocation constraints of optical network,task processing methods of edge servers,and classification of edge computing tasks.On this basis,a mathematical model of scenario and tasks is established.Based on the above mathematical model,a Markov Decision Process for the joint allocation of communication and computing resources in the metro optical networks is constructed.Afterwards,a reinforcement learning scheme was designed to handle the Markov Decision Process.By offloading services between edge servers,the solution can increase the capacity of metro optical networks for services on the basis of existing resources.In the comparative experiment,the reinforcement learning scheme reduced the blocking probability of services by 53%,achieving the expected increase in network capacity.In order to improve the adaptability of the proposed scheme and meet the requirements for the adaptability and generalization of the scheme in the field of optical network control,this paper introduces transfer learning to enable agents to realize knowledge reuse and rapid transfer in different networks.Based on software defined networks,two deployment schemes are designed for the intelligent agent,namely single agent scheme and multi agent scheme,to meet the different requirements of different optical networks for scheme generalization and decision performance.The experimental results show that transfer learning reduces the training amount required by the intelligent algorithm to converge in the new environment by more than 60%.The experimental results also verified that the Single-Agent scheme has better generalization,while the Multi-Agent scheme can increase the network capacity to a higher level.