| With the development of mobile Internet,the massive data traffic caused by the rapid increase of mobile terminal user devices has brought great challenges to the network communication system,resulting in network congestion and increased response time.Moving edge cache technology can alleviate network congestion and reduce user response time.However,how to design and optimize the cache strategy,improve the cache efficiency and improve the quality of user experience is still faced with many challenges.In order to improve the quality of user experience,this paper uses reinforcement learning,collaborative caching between user devices and microbase stations to optimize the caching strategy and improve the caching efficiency.The main research work of this paper is divided into the following aspects:(1)Research on Edge cache resource Selection Strategy based on multi-arm slot machine.In the multi-user and multi-edge server scenario,the uncertainty of user requests and base station cache is considered,and a large number of user requests per unit time may lead to network congestion.In order to alleviate the network congestion problem,this paper introduces the M/M/1 queueing model in queueing theory,and uses the queueing model to simulate the process of user sending request service.Since users choose different edge servers to obtain cache resources at different costs,UCB algorithm in reinforcement learning is used to update the edge base station selection strategy.Combined with the exploration and utilization ideas in reinforcement learning,the edge cache selection strategy based on multi-arm slot machine was proposed according to the return value obtained each time the user selected the edge base station and constantly updated,aiming at maximizing the total return value of the user.Experiments were conducted on the EUA dataset in Melbourne Central Business District to verify the effectiveness of the proposed algorithm in improving the total return value of user equipment and reducing the total delay of user requests.(2)Research on caching strategy based on collaboration between user devices and microbase stations.In this paper,a three-layer heterogeneous network model is constructed and the joint cache optimization problem between the user device side and the micro-base station side is studied.In the network model,user content request can be satisfied through the user device local cache,communication sharing transmission between devices and microbase station transmission.By considering the delay and energy consumption optimization brought by different delivery methods,this paper combined the delay and energy consumption problem into a non-convex optimization problem with maximum total efficiency,and adopted genetic algorithm to solve it.By conducting experiments on EUA data sets,it is verified that the proposed joint caching strategy of user devices and microbase stations can maximize the total efficiency of the system and improve the quality of user experience. |
