Research On Online Computing Offloading Strategy Of MEC Based On Deep Reinforcement Learning

With the rapid development of wireless communication and mobile applications,there are more and more kinds of mobile smart devices and more powerful functions,such as Io T,5G,UAV,Smart Home,VR and so on.These new applications put forward higher requirements on the computing power,battery life and energy consumption of mobile devices.However,resources such as computing power and storage of mobile devices are often lacking,which has become a key problem in current mobile device applications.To solve this problem,some experts proposed MEC,which can effectively solve the problem of insufficient resources of mobile devices by deploying servers near mobile devices to provide powerful computing services for mobile devices.In order to improve the battery life of mobile devices and continuously provide energy for mobile devices,wireless power transmission technology is introduced in MEC,which can provide mobile devices with the energy required for computing and offloading,and can convert the remaining energy into electrical energy to charge mobile devices.This paper studies the goal of maximizing the residual energy of the user devices in the wireless power supply MEC,which can be expressed as the problem of maximizing the residual energy of the user devices.At present,to solve this problem,it is often converted into MIP for modeling and solving,but the traditional offline state-based numerical optimization method is difficult to make timely and correct unloading decisions in a fast fading wireless channel environment.In order to overcome the environment of rapid fading of wireless channels,this paper proposes a DRLO algorithm,which provides an optimal binary offloading strategy for user devices,so as to maximize the acquisition of wireless transmission energy and minimize the energy consumption.Consume user devices energy to improve the battery life of the mobile device and provide users with a better service experience.The binary offloading decision is that the computing tasks are either all executed on the wireless devices,or all the computing tasks are offloaded to the MEC server for computing and then the results are returned.The main work of this paper is as follows:(1)Establish a system model.For a mobile edge server with multiple antennas and N user devices with single-antenna,a system model is established based on TDMA communication protocol by combining MEC with wireless power transmission technology.In order to achieve the research objective of maximizing the overall residual energy of the user devices,an optimization scheme of jointly optimize the user devices offloading decision,wireless energy transmission time,task offloading time and other resources is proposed.(2)Problem analysis and solution.The formal problem of maximizing the residual energy of user devices is a non-convex problem,which is difficult to solve.However,it is found by analysis that when the offloading decision is given,it can be transformed into a convex problem and solved by convex optimization technology.Therefore,we split the target problem into two sub-problems,namely the time resource allocation problem and the task offloading problem.For the time resource allocation problem,the convex optimization method can be used to solve the relevant optimal parameters;for the task offloading problem,based on the deep reinforcement learning model,a DRLO algorithm is constructed with the input as the wireless channel gain and the output as the maximum residual energy of the user equipment,so as to obtain a MEC online computing offloading strategy in a wireless fading environment can maximize the remaining energy of the user devices,prolong the working time of the devices,and improve the user experience.(3)Analysis of experimental results.The simulation results show that the DRLO algorithm proposed in this paper for the research goal of maximizing the residual energy of MEC user devices can provide users with an online computing offloading strategy better than other traditional benchmark schemes,especially when the user equipment exits the system due to power shortage or failure,or the system accesses new equipment,it can make timely and automatic adjustment without manual participation,and has very good adaptability.