Research And Implementation Of DRL-based Resource Allocation In Network Virtualization

With the continuous increase in demand for internet services by users,the existing network services are in urgent need of upgrades.However,traditional service improvements require a significant amount of manpower and resources.Each update to the network infrastructure incurs substantial procurement expenses and management costs.Clearly,this unsustainable development approach does not align with the current technological goals.Therefore,a more energy-efficient and efficient upgrade approach should be sought.With the development of virtualization technology and standard industrial servers,Network Function Virtualization(NFV)has been proposed as a new network architecture.It deploys dedicated devices in software form,abstracting the hardware’s resource capabilities,and replacing the traditional hardware network functions.NFV has become a favored technology among operators,equipment vendors,and service providers.One of the key challenges in virtualized networks is the resource allocation problem,specifically how to deploy network services in the network infrastructure.One crucial issue is how to map virtual networks onto the physical network,known as the Virtual Network Embedding(VNE)problem,which has been proven to be NP-hard.Existing research has explored three approaches: exact solutions,heuristic algorithms,and reinforcement learning.Among them,reinforcement learning,particularly deep reinforcement learning,has demonstrated significant advantages.It can gradually learn better strategies by interacting with the environment and leverage deep neural networks to handle largescale state-action spaces.Based on deep reinforcement learning,this thesis proposes a VNE solution.Experimental results show that the proposed algorithm exhibits certain advantages in different control groups.The main contributions of this thesis are as follows:1)Using action scheduling rules and a dual-experience replay pool to balance the decision-making of the intelligent agent.In deep reinforcement learning,initial action exploration by intelligent agents is typically done blindly and randomly.To avoid this issue,heuristic action generation is introduced to guide the decisionmaking of the intelligent agent.2)Graph SAGE is used to implement feature extraction of physical network topology.Traditional convolutional neural networks(CNN)are unable to handle nonEuclidean data feature extraction.While graph convolutional networks(GCN)can perform feature extraction on graph data,they suffer from scalability issues.In contrast,Graph SAGE can generalize unknown nodes to adapt to the dynamic changes of the physical network topology.3)This thesis also implements a VNE simulation experiment prototype system that can generate network topologies for comparative experiments with different algorithms.