Virtual Function Component Scaling Optimization Based On Deep Reinforcement Learning In Radio Access Network

Network function virtualization reduces the cost of Telecommunications Service Providers-by deploying service function chain composed of virtualization network function components instead of deploying network functions on physical dedicated equipment.Due to complexity of service function chains and dynamic changes in traffic patterns,placement optimization of virtual network function components has become one of the main challenges in NFV.Reinforcement learning is an effective approach to solve optimization problems.Therefore,optimal placement of virtual network function components using reinforcement learning has become a research issue.Deep reinforcement learning fits the action value function by means of function approximation,which solves the problem of huge state and action space.Hierarchical reinforcement learning presents potential options for agents to perform more structured exploration by decomposing tasks into several sub goals.Based on the hierarchical deep reinforcement learning,this thesis studies the auto-scaling optimization problem for virtual function components in radio access network.The research contents and main innovations are summarized as follows:1.The key technology of virtual function component scaling optimization based on hierarchical deep reinforcement learning is investigated,and the research progress of virtual function component automatic scaling optimization is analyzed.2.The auto-scaling optimization problem of virtual network function components is modeled as an optimization problem,which aims to minimize weighted sum of average waiting time,average queue length and VNF instance cost.Considering the problem of QoS degradation and traffic congestion caused by service requests,an auto-scaling optimization algorithm called VFC-AS-DLO is proposed,by combing LSTM,deep reinforcement learning and Option.The performance of VFC-AS-DLO is evaluated by simulation,and its performance is verified compared with benchmark algorithms.3.Considering that the action space search of the agent is too large due to the use of the greedy strategy,based on hierarchical reinforcement learning,a virtual function component scaling optimization algorithm(VFC-AS-HDQN)based on H-DQN is proposed.The performance of VFC-AS-HDQN is simulated and evaluated.Compared with the DQN algorithm,VFC-AS-HDQN algorithm has better performance in terms of system utility.