Deep Reinforcement Learning-based Optimization Of Offloading Policies For Multi-user Mobile Edge Computing Tasks

With the development of Io T,5G and other technologies,the amount of new data added daily is exponentially exploding,and these data from various emerging application scenarios such as telemedicine,smart car driving,smart cities,etc.,put more stringent requirements on the service URLLC(Ultra Reliable Low Latency Communication).The requirements are more demanding.To better meet these requirements,Mobile Edge Computing(MEC)has emerged.A very important part of mobile edge computing is task offloading.In the face of today s requirements of big data,low latency,and high reliable quality of service assurance,the selection and efficiency of task offloading strategies are particularly important.Therefore,how to choose task offloading strategy,how to weigh the problem of delay and energy consumption in offloading,and how to balance the computation of each part in a system with limited resources,i.e.,load balancing,also become a series of urgent problems to be solved.This paper introduces the development history of mobile edge computing and the current research status of tradeoff delay and energy consumption and load balancing.It also focuses on how to make it adaptive to trade-off these aspects.To achieve the effect that can be adaptive,this paper combines deep learning,reinforcement learning,and policy gradients to propose the DQN-HDCS algorithm and MOPERDQN algorithm based on Deep Reinforcement Learning(DRL).The main work of the paper is as follows.(1)A MEC system consisting of multiple mobile users is studied,where both the task arrival process and the wireless channel are set to be random,as a way to calculate the long-term average computational cost of different choices of task offloading policies on latency and power consumption,and to minimize them by the algorithm.To this end,a Markov decision process is established for the sub-scenario based on its next state that is only related to the current state,and a global adaptive algorithm is designed for decentralized adaptive computing task offloading policies based on deep reinforcement learning.Specifically,in order to handle a continuous action space and to act dynamically on the whole system,the optimal task offloading policy selection is learned independently and efficiently based on the feedback provided by each terminal in the system.Based on the results provided by the simulation experiments,it can be seen that in a multi-user scenario,by continuously learning at each user,the system can better trade-off the latency and power consumption to reach the minimum value.(2)The load balancing problem of each computing device in the system in the case of a multi-user heterogeneous MEC network is investigated.It consists of three layers:mobile users,edge computing nodes and cloud data center,each of which includes heterogeneous computing devices.In this scenario,we model the load balancing,the intelligent agent receives load feedback from each node and learns independently,and introduce a special experience replay method,i.e.,"failure is the mother of success",to reward the actions that do not reach the target state,thus We propose the MOPERDQN algorithm based on post-hoc experience replay to accelerate the learning efficiency of the intelligence and guide it to the correct learning direction in order to achieve the global load optimal policy.Experiments show that this adaptive global dynamic learning algorithm improves the learning rate and balances the load of each computing node.