Research On Dynamic Pricing Strategy In Edge Computing Based On Two-layer Deep Reinforcement Learning

The large-scale edge computing system consists of many edge computing servers deployed in various locations in the system.The edge computing server helps users by providing paid task offloading services to complete computing tasks efficiently.The pricing scheme of edge computing service can not only determine the total revenue of edge computing service providers,but also play an important role in guiding the task offloading decision of users.Considering the movement trajectory and the uncertainty of users’ task offloading preferences,designing intelligent dynamic pricing algorithm to maximize the total revenue of edge computing service providers is a very challenging and important topic.The main work of this thesis is as follows:(1)In order to make the best pricing decision in real time with comprehensively considering users’ dynamics and edge computing server load,this thesis designs a dynamic pricing framework of edge computing services based on reinforcement learning,and models the dynamic pricing process of a single edge computing server as a Markov decision process,on this basis,a dynamic pricing algorithm based on Dueling Double Deep Q Network(D3QN)and a dynamic pricing algorithm based on Soft ActorCritic(SAC)are proposed for discrete pricing actions and continuous pricing actions respectively.(2)Due to the diversity of load patterns of edge computing servers in large-scale edge computing systems,and each edge computing server needs to serve users with different task offloading preferences,in order to further improve the total revenue and computing resource utilization of edge computing service providers,this thesis designs a joint task scheduling framework based on reinforcement learning(RLJS).Specifically,we first use the data-driven method to divide the edge computing servers in the system into groups according to their load patterns,and then a joint task scheduling algorithm based on D3 QN is proposed to intelligently schedule computing tasks among groups in real time according to the load and service price of each edge computing server group.Simulation results demonstrate the efficacy of RLJS in improving the total revenue of the system provider and the QoS of users.