Research On Multi-layer Virtual Network Embedding In SDN Based On Reinforcement Learning

With the rapid development of cloud computing,Internet of Things,and 5G,the traditional network architecture becomes much difficult to deploy new network technologies and protocols.The combination of Software Defined Network(SDN)and Network Virtualization(NV)technologies is considered as an effective way to overcome current network ossification and promote future network innovation.NV introduces virtualization into the network,allowing multiple virtual networks to run on the same physical network.The core idea of NV is the decoupling of software-based virtual networks from hardware-based physical networks,so it is much easier to implement NV in SDN.Virtual Network Embedding(VNE)is a key issue in NV,dealing with how to allocate physical resources for multiple virtual network requests(VNR)with topology and resource constraints.As the widely deployment of SDN in large datacenter or wide area networks and the emergence of large tenants with hierarchical organizational structures,it is a new requirement and possibility to provide multi-layer encapsulated virtual network services for tenants.Therefore,VNE algorithms should be flexible enough to map VNRs onto the physical network and upper VNRs onto their corresponding substrate VNRs.Aiming at the above problems,this thesis proposes a multi-layer dynamic collaborative VNE algorithm in SDN based on reinforcement learning.It adopts a two-step idea to map upper or substrate VNRs,which will map all virtual nodes at first and then map virtual links.In addition,in order to improve the flexibility of the algorithm,a model named MLRL-Model is introduced into the node mapping stage,which is based on reinforcement learning.Before mapping each VNR,the node mapping policy will be updated in real time by training MLRL-Model firstly,overcoming the simplicity of mapping strategy and lack of flexibility in the whole mapping process in existing algorithms.Although the topology and the total resources of the physical network are static,the mapping result of the substrate VNR can be adjusted dynamically.Therefore,when a VNR that has been successfully accepted by the physical network cannot meet the mapping requirement of its upper request,it can accept this upper request by adjusting its original mapping result.In order to improve the ability of substrate VNRs to accept upper requests as much as possible while minimizing the impact on the stability of network virtualization environment,this thesis further proposes a dynamic mapping mechanism,making the resources of physical network and substrate virtual networks collaborative.It gives priority to increasing the resource capacity of particular substrate virtual nodes(or links)or migrating them to other physical nodes(or links)with remapping the entire substrate virtual network as the complement operation.The experiments show that the proposed algorithm not only has a good performance in single-layer VNE scenario,which improves its acceptance ratio,average node and link utilization of physical network,but also makes under virtual networks accept much more upper requests and improves the total acceptance ratio in multi-layer VNE scenario.Firstly,this thesis analyzes the research status of SDN and VNE and points out the problems of existing algorithms.Then,this thesis defines multi-layer VNE in SDN and elaborates the proposed multi-layer dynamic VNE algorithm.Finally,this thesis describes the design and implementation of experimental simulation and evaluates the performance of the proposed algorithm.