Research On Content Delivery Mechanism In Edge Computing Network Based On Reinforcement Learning

With the development of the Internet of Things technology and the official commercialization of the fifth-generation communication technology(5G),network traffic has exploded,and real-time services such as unmanned driving and smart grid have put forward higher requirements for delay.The powerful computing power of the cloud computing model can reduce computing latency,but the transfer of massive data to the cloud causes network congestion and increases computing response latency.The edge computing model that is geographically closer to the user can reduce data transmission delay and increase the speed of computing response.However,the time gains brought by edge computing when processing resource-intensive applications are extremely small and need to be combined with the powerful resource advantages of the cloud.At the same time,the computing performance of user terminal equipment continues to improve,so that the user side can also participate in edge computing to share part of the computing tasks,forming a new type of converged cloud-edge-end edge computing network.There are still many challenges in the content delivery of the current cloud-side-end edge computing network.First,the cloud server sends the modules required for computing tasks to the edge server,so that the edge server has computing power and processes real-time computing content to reduce computing delay.However,dynamic changes in network traffic will cause network congestion,which will increase the network delay for cloud servers to send computing content to edge servers.How to plan the route between the cloud and edge to improve the efficiency of content delivery is an urgent problem to be solved.Collaborative reinforcement learning can learn environmental information in real time and can adapt to the dynamically changing network environment.Therefore,it is a challenge to use collaborative reinforcement learning methods to build routes between cloud edges to reduce transmission delay.Secondly,considering the user's own computing power and the computing resources of the remote cloud server,the user can split the content that needs to be calculated and send it to the edge server and the cloud server respectively,so that the user,the edge,and the cloud can perform parallel computing to improve computing efficiency.However,limited bandwidth and computing resources will affect content delivery delays and computing delays.How to split computing content and send it to the cloud and edge to improve overall computing efficiency is an urgent problem to be solved.Deep reinforcement learning combines the decision-making ability of reinforcement learning and the perception ability of deep learning,and can handle decision-making problems in high-dimensional state spaces.Therefore,using deep reinforcement learning methods to solve the optimal content delivery strategy to improve the overall computational efficiency is a challenging problem.In summary,this article considers the dynamics of the network and the limited computing resources,and proposes an edge computing network content delivery mechanism based on reinforcement learning.The specific work is as follows:(1)In order to reduce the content delivery delay between cloud and edge in edge computing network,this paper proposes a cloud-edge content delivery method based on collaborative reinforcement learning.First,a collaborative reinforcement learning framework based on the edge computing network is constructed.The V-value feedback model is used to capture the dynamic characteristics of network traffic,and the Q-value is used to quantify the transmission efficiency of the node.And on the basis of this framework,a delivery tree construction algorithm based on collaborative reinforcement learning is proposed,and the delivery path from cloud server to edge server is constructed through this algorithm.Through numerical simulation,it is found that the cloud-to-edge delivery path constructed by this method can effectively improve the content delivery efficiency.(2)In order to improve the overall computing efficiency of edge computing network,this paper proposes an edge computing network content delivery method based on deep Q-learning.First,build a content processing model to calculate the delay and energy consumption of different delivery strategies,and define the weighted sum of the two as the total cost of calculation.Then a content delivery model based on deep Q-learning is proposed,and the optimal delivery strategy is solved through this model.Numerical simulation results show that this method can effectively reduce the calculation cost of the system.