Reinforcement Learning Based Network Utility Optimization For Network Function Virtualization

Network Function Virtualization(NFV)emerges as a new networking service provision paradigm.The equipment in network environment are quite different in terms of computing power,storage resources and server topology.As a result,the resources shall be provisioned in an adaptive and hybrid way according to the communication demands.Traditional model-based network utility optimization methods are limited in practical application due to the involvement of assumptions or preconditions.Therefore,it is always desirable to design optimization methods that do not require any prior knowledge.In this thesis,we investigate how to improve the performance efficiency of service function chain(SFC)in heterogeneous network environment,targeting at two kinds of SFC consisting of,i.e.ordered and unordered VNFs,respectively.For SFCs with unordered network functions,there is no sequential or data dependency between network functions.The VNFs can be freely deployed on any server.At runtime,the time-variability and mobility of user requests,as well as the VNF locations have huge impact on service performance.How to eff-iciently migrate the VNFs according to the user request dynamics therefore is critical to the network performance.To this end,we apply Deep Q Network(DQN)to realize the service migration management algorithm in reaction to user mobility so as to reduce service cost and improve service performance.Experiment results show that the DQN-based algorithm can automatically learn the time varying user request pattern and make efficient service migration decisions accordingly.When the VNFs in an SFC must follow strictly order,it is necessary and significant to jointly consider network flow scheduling and VNF orchestration for maximizing network utility.Aiming at this problem,a customized algorithm based on Deep Deterministic Policy Gradients(DDPG)is designed to realize intelligent VNF orchestration and network flow scheduling under the consideration of end-to-end delay and various costs.Extensive experiments verify that the performance of the customized DDPG algorithm outperforms both the model-based optimization algorithm and the non-customized DDPG algorithm.