Research On Deep Reinforcement Learning Based Computation Offloading Mechanism

In recent years,technologies such as Internet of Things and cloud computing have been widely used.The centralized ability of cloud computing has many shortcomings at the edge of the network,which affects user experience.Edge computing has emerged as the times require,it closes to the source of things or data on the edge of the network,integrates core capabilities of network,computing,storage,and applications,provides edge intelligent services nearby,and meets the key needs of industry data in agile connection,real-time business,and data optimization.Aiming at the existing problem in the adaptation of offloading strategies,the waste of cloud resources,the mobility of user terminals,and the relevance among tasks,the main innovative contributions of this thesis include the following three aspects:1)Deep reinforcement learning based edge computation offloading mechanism: Aiming at the requirement of several delay-sensitive and computation-intensive tasks,an edge computing offloading mechanism with coordinated delay and energy consumption is proposed.Specifically,a minimization problem of the weighted sum of task completion time and energy consumption is formulated to achieve the efficient task offloading computing by jointly optimizing the allocation of CPU computing power and bandwidth.At the same time,a deep reinforcement learning based edge computing offloading algorithm(DRL-ECO)is proposed,this algorithm integrates the target function of the edge computation offloading system and the target network,employs the experience playback mechanism,and comprehensively optimizes the network resources based on the optimal offloading strategy to further improve the network efficiency.Finally,the extensive simulation results show that the proposed DRL-ECO can quickly obtain the optimal offloading strategy with lower computing resource requirement and significantly reduce network costs as compared with the traditional optimization methods.2)Deep reinforcement learning based cloud-edge collaborative computation offloading mechanism: Aiming at the limitation of single edge node's computation and storage resources and the demand of efficient computing services in big data scenario,this thesis proposes a deep reinforcement learning based cloud-edge collaborative computation offloading mechanism.Specifically,based on the comprehensive consideration of computing resources,bandwidth and offloading policy,an optimization problem is formulated to minimize the weight sum of all user tasks' execution delay and energy consumption.A deep reinforcement learning based asynchronous cloudedge collaborative algorithm(DRL-AC3O)is proposed to solve such optimization problem.According to environment differences of edge nodes in the edge cloud,this algorithm can adaptively adjust offloading policy to effectively reduce the correlation of samples.At the same time,an asynchronous multi-threaded method is employed to replace the traditional experience playback mechanism,which avoids the high memory overhead of edge cloud.In addition,a cloud-edge advantage function with better reward feedback for offloading action is constructed to replace the traditional loss function,it makes DRL-AC3 O can obtain the optimal offloading decision more quickly.Finally,the extensive simulation results show that the proposed DRL-AC3 O algorithm has the characteristics of fast convergence rate and high robustness,and its optimal offloading policy closely approximates to the solution of greedy algorithm with the lowest computation cost.3)Deep reinforcement learning based cloud-edge collaborative mobile computation offloading mechanism: Due to the limited coverage of static edge servers and the traditional edge computing technologies do not perform well in today's environment,in order to adapt to diverse needs,a deep reinforcement learning based cloud-edge collaborative mobile computing offloading mechanism is proposed.Specifically,a three-layer network model of digital twins and a decentralized network of task resources are first constructed to handle the mobility of user terminals and the relevance of tasks.And then based on the comprehensive consideration of mobility,associated tasks,computing resources and offloading decisions,an optimization problem is formulated to minimize the weighted sum of the execution delay and energy consumption of all tasks for all users.Based on this optimization problem,a deep reinforcement learning based cloud-edge collaborative mobile computation offloading algorithm(DRL-CCMCO)is proposed.Based on the differences of each edge cloud,this algorithm sets the priority of the shared experience pool,and selects the most effective experience samples to complete better learning and training.It also utilizes a distributed learning method to learn the probability of an approximate reward distribution,and optimizes network parameters through cloud-edge collaboration mannerfor achieving the optimal offloading decision faster.Finally,a large number of simulation results show that the proposed algorithm has the characteristics of fast convergence and high stability,and it can obtain the optimal offloading decision with the lowest total cost.