Research On Task Offloading And Resource Allocation Strategy In Multiple Edge Servers Based On Reinforcement Learning

In the environment with multiple edge servers,users’ demands arrive dynamically and the tasks can be offloaded to the edge servers for processing.In this thesis,task offloading and resource allocation in the environment with multiple edge servers are investigated to maximize the number of served users and minimize the average energy consumption of all tasks while meeting the task delay requirements.Considering that some edge servers are centrally controlled or independent in the actual mobile edge computing environment,in this thesis,we investigate this problem under the setting with a central controller and multiple edge servers cooperating with each other respectively.The research of this thesis is as follows:(1)The setting of task offloading and resource allocation is presented.Firstly,the setting of the edge server and the task is introduced,then the process of task execution on the user device and offloading to the edge server is described.Finally,the optimization goal of task offloading and resource allocation of this thesis is given.(2)The task offloading and resource allocation under the management of the controller are investigated.Considering that in the actual environment,multiple edge servers are centrally managed by a controller,and the coverage of edge servers partially overlaps with each other,task offloading and resource allocation in this scenario is studied.The controller can make a reasonable decision based on the task information and the state information of multiple edge servers.As the problem involves discrete and continuous mixed action space,we model this problem as a Parameterized Markov Decision Process,and then use the deep reinforcement learning algorithm P-DQN(Parameterized Deep Q-Network)to solve it.Finally,we run experiments to evaluate our algorithm against the other four algorithms,Random,Greedy,NO,and HTR.The experimental results show that our algorithm can maximize the number of served users and minimize the average energy consumption of all tasks under the condition of meeting the task delay requirements.(3)The task offloading and resource allocation in the environment with multiple edge servers is further analyzed.In this scenario,there exists no controller to manage multiple edge servers.Each server is an independent agent,and each agent can make the decision independently based on its local observation.In the service coverage overlapping area of edge servers,the task offloading and resource allocation strategy of an edge server will affect the interests of the other edge server.Therefore,it is a cooperative problem,and edge servers need to collaborate with each other.In this thesis,we model the problem as a Partially Observable Markov Game model(POMG)and use the multi-agent reinforcement learning algorithm I-PDQN(Independent Parameterized Deep Q-Network)to solve it.Compared with Random-C,Greedy-C,NO-C,and HTR-C algorithms,we find that the proposed algorithm can effectively improve the total number of served users and minimize the average energy consumption of all tasks under different user numbers,edge server computing capacity,and edge server bandwidth parameters.In summary,we analyze the task offloading and resource allocation in the setting with multiple edge servers.Considering the scenarios with a controller and multiple edge servers’ cooperation,task offloading and resource allocation problem is modeled as a Parameterized Markov Decision Process,and Partially Observable Markov Game model respectively,and then reinforcement learning algorithm P-DQN and multi-agent reinforcement learning algorithm I-PDQN are used to design the task offloading and resource allocation strategy,to maximize the number of served users and reduce the average energy consumption of all tasks while meeting the task delay requirements.