| Mobile edge computing can provide computing service close to users to reduce delay and improve service quality,and different offloading strategy will greatly affect efficiency.Therefore,formulating an appropriate and efficient offloading strategy is very crucial.Deep reinforcement learning can solve the problem of mobile edge computing offloading effectively by training the agent in interacting with the environment to learn strategy which can achieve long-term benefits.Therefore,based on deep reinforcement learning,the offloading problem is studied.The main research works are as follows:Firstly,the centralized offloading strategy in the scene of multi-user with single server is considered.Aiming at the problem that the continuous offloading strategy is easy to converge to the local optimal solution,a continuous offloading strategy TOSAC with deep reinforcement learning based on SAC is proposed.By using the information entropy in the offloading strategy,the exploration ability of agent is improved and the local optimal solution is effectively avoided.The optimization goal of this strategy is to minimize the delay and energy consumption of terminal equipment,and the local computing model,edge computing model and the problem description are given.Then the Markov decision process model is established and the offloading strategy steps are given.Finally,the simulation experiment shows that the delay of TOSAC offloading strategy is22.31% lower than that of EOC offloading strategy,and the comprehensive performance reward is9.52% higher on average;compared with TOGA offloading strategy,TOSAC offloading strategy reduces the delay by 10.37% on average and improves the comprehensive performance reward by7.36% on average;compared with DDPG offloading strategy,TOSAC offloading strategy reduces the delay by 7.69% on average and improves the comprehensive performance reward by 3.99% on average.Secondly,based on the scene of multi-user with single server,considering the multi-user with multi-server offloading problem,TOSAC is extended to this scene,and a multi-server collaborative offloading strategy MSTOSAC is proposed.In this scene,it is necessary to determine not only the offloading proportion,local calculation frequency and transmission power of each terminal device,but also the target mobile edge computing server to be offloaded.Finally,the simulation experiment shows that MSTOSAC can reduce delay and energy consumption and improve comprehensive efficiency with the increase of mobile edge computing server.Compared with the MEOC offloading strategy,the delay is reduced by 19.7% on average,and the comprehensive performance reward is increased by 7.3% on average;compared with the MDDPG offloading strategy,MSTOSAC offloading strategy reduces the delay by 13.4% on average and improves the comprehensive performance reward by 4.5% on average.Thirdly,based on the above offloading strategies and corresponding technologies,a mobile edge computing offloading management system based on deep reinforcement learning is designed and implemented.The system designs and implements terminal device management module,agent management module,edge computing node management module,user management module and login and registration module.The system test results show that the system can realize the management of terminal devices,agents and edge computing nodes,and help to improve the efficiency of user management. |