A Study On Optimal Computing Offloading Policies Based On Reinforcement Learning In Edge Computing

Multi-Access Edge Computing(MEC)is an emerging and important cloud computing architecture in 5G and future networks,designed to extend cloud services to the network edge.End-user devices can offload application tasks to MEC for computation to reduce application service latency and energy consumption and bring better quality of service(Qo S).Therefore,it is crucial to optimize the MEC task offloading policies in order to minimize the application task latency and energy consumption.However,the complexity and variability of MEC system environments and end-application tasks lead to the fact that optimizing MEC task offloading is not an easy job.This thesis investigates the existing related research and finds that the traditional optimization methods are usually simplified mathematical models and specific heuristic algorithms based on expert knowledge.However,when confronted with dynamic MEC scenarios,it requires a lot of manpower and expertise to redesign the models and adapt the heuristics,which is time-consuming and labor-intensive.Besides,there are also studies on using machine learning to do MEC task offloading policies optimization,but they still have some problems: first,the optimization objective is relatively single,with task latency as the main focus;second,similar to heuristic methods,they cannot better adapt to the complex and changing MEC system environment.Considering the restrictions of these methods,this thesis proposes to use deep reinforcement learning(DRL)methods to address the task offloading policies optimization problem of MEC systems.This thesis first analyzes the task with fine-grained to directed acyclic graph(DAG)modeling,fully consider the task complexity and internal dependencies,and design a DAG topology priority algorithm(DTPA)to enrich the information of DAG task sequences,which provides better information input for DRL learning training and enables the system to learn the best unloading strategy by itself.The MEC computational offloading problem is converted into a Markovian decision process,and a recursive neural network is designed to encode and decode the DAG task sequence to do the mapping from task input to offloading strategy,and the DRL method is applied to train the optimal offloading policies.Quantifying the comprehensive profit of task delay-energy and using it as the optimization objective,experimental analysis supports the effectiveness of the DRL-based approach on task offloading for the key optimization problem of MEC systems.The experimental results in different scenarios reveal that the comprehensive profit of task delay-energy achieved by the DRL-based fine-grained tasks offloading strategy with DAG-priority(DFTOSD)in this thesis are about 0.1-0.3 higher than those achieved by existing heuristics,general reinforcement learning methods,and greedy algorithms.Based on this,the key factor that affects the comprehensive profit of task delayenergy of the MEC task offloading policies,the network bandwidth,is further discussed and analyzed.To this end,this thesis also integrates deep reinforcement learning to optimize the network bandwidth problem caused by the number of access terminals under the MEC system.Simulation experiments show that the DRL network optimization algorithm(DRL-CW)proposed in this thesis can better reduce the negative impact of increasing the number of access devices on the network bandwidth,and can improve the performance by about 36% relative to the default Exponential Back-off Algorithm(EBA).Finally,based on DFTOSD and DRL-CW,this thesis proposes a deep reinforcement learning-based joint network optimization MEC task offloading policies scheme(D-DFTOSD).Experiments show that D-DFTOSD can obtain about double the comprehensive profit of task delay-energy compared with DFTOSD in the MEC system environment with a larger number of access devices.