Research On Service Migration Algorithm For Edge Computing Based On Reinforcement Learning

Edge Computing can sink computing capacity and storage capacity to the edge of the network.Compared with Cloud Computing,it can provide high reliability and low latency service,so that many time-sensitive services can be applied.These services embedded in the user equipment can be connected to the Edge Computing Server closest to it and the user can get a better experience.However,user mobility and limited coverage of Edge Computing Server result in service discontinuity and reduce Quality of Service.We can apply service migration to solve this problem.In service migration,how to choose the optimal migration strategy and communication strategy is a key issue.Aiming at the problem of service discontinuity caused by user mobility and limited coverage of Edge Computing Server for Edge computing,this paper studies single-user scenario and multi-user scenario,and proposes service migration algorithms based on reinforcement learning.First,a service migration algorithm based on reinforcement learning in a single-user scenario is proposed.Aiming at the relative simple problem of the current service migration model,a mathematical model of service migration in a complex environment is established,taking into account constraints such as resource capacity,link capacity and delay.The model is transformed into a form that reinforcement learning can solve.For this scenario,define state,action and reward in detail.We innovatively propose a service migration algorithm based on Q-learning and Deep Q Network.In a long-term process,the optimal migration strategy and communication strategy of the user's corresponding virtual machine are obtained.Simulation results show that under different system parameters,the proposed algorithm can achieve the best results compared with other algorithms.Second,a service migration algorithm based on reinforcement learning and game theory in a multi-user scenario is proposed.A mathematical model for multi-user service migration in a complex environment is constructed.This model is transformed into a mixed-task dynamic random game problem in multi-agent reinforcement learning,solving the long-term Nash equilibrium solution of multiple agents in mixed-task dynamic random games.In the process of solving the Nash equilibrium solution of the game at each stage of the dynamic game,an intelligent algorithm for solving the Nash equilibrium of n-person non-cooperative games is designed,and a Nash equilibrium solving algorithm based on the maximum expected sum of genetic algorithms is proposed.For this scenario,the joint state,joint actions and rewards of multiple users' corresponding virtual machines are defined in detail.We innovatively propose a service migration algorithm based on Nash Q-learning.A joint optimal migration strategy and communication strategy for multiple users' corresponding virtual machines are obtained in a long-term process.The simulation results show that compared with other algorithms,users'corresponding virtual machines can better cooperate in our proposed algorithm.Under the premise of ensuring the completion of the task,the whole system achieves the best results.In short,this paper studies the service migration problem in single-user and multi-user scenarios.In a single-user scenario,the overall cost is minimized.In a multi-user scenario,the overall cost is minimized on the premise of achieving a long-term Nash equilibrium solution.The proposed algorithms provide a solution for migration strategy and communication strategy of users'corresponding virtual machines for Edge Computing.