Research On Edge Cooperative Caching Strategy Based On Multi-Agent Reinforcement Learning

The popularization of the 5G has greatly improved the quality of mobile communication.It has greatly improved the user experience and affected many industries with extremely low latency,large bandwidth,ultra-high speed and more connections.According to the traditional way,data is ultimately obtained from the cloud center.A large part of mobile data can be reused,caching this part of data in advance on the side closer to the user can effectively reduce the burden of the cloud center and provide better services.The cooperation of edge nodes is an effective way to improve the edge performance,but current research mostly assumes that edge nodes are in a fully cooperative state,lacking effective cooperative awareness,making it difficult to apply in large-scale edge clusters.Therefore,based on modeling the edge system as a multi-agent system,this dissertation studies the cooperative caching strategy under the distributed deployment of base stations,and uses multi-agent reinforcement learning to make decisions on cache placement.The main research contents of this dissertation are as follows:(1)Aiming at the balance between content diversity and transmission diversity,a neighbouring-aware edge cooperation caching strategy based on multi-agent reinforcement learning is proposed.The strategy fully analyzes the balance between content diversity and transmission diversity in base station cooperation,and a Neighbouring-Aware Multi-Agent Cooperative Caching(NAMACC)placement strategy is proposed.The placement strategy is optimized by Neighboring-Aware Upper Confidence Bound(NAUCB)algorithm.NAUCB considers the benefits of cache placement to local base station and its neighbor base stations.Since a large part of user requests come from a small number of contents,the strategy gives priority to meeting the requirements of most local users and ensuring the quality of service of local users,at the same time,it senses the edge system benefits that can be brought by assisting neighbouring base stations,and caches some content that can bring higher edge system performance improvement through cooperation.The experiment shows that the proposed cooperative caching strategy reduces the latency by at least 10% compared with the other strategy.(2)Aiming at the problem of elastic resources in the edge cooperation cache,the multi-agent reinforcement learning Elastic Cooperative Edge Cache(ECEC)framework based on Gated Recurrent Unit(GRU)and Counterfactual Multi-Agent(COMA)policy gradients is proposed to jointly optimize the request delay and lease cost in the cooperation of the base stations.The framework first uses GRU to predict the user request rate at the next moment,and solves the asynchrony between the cache decision and the user request pattern perception.Then,the COMA is used to make the next time cache placement decision.In the process of decision making,in order to reduce the size of input and output space and reduce unnecessary noise information input,the branch COMA technique is proposed.The final experiment shows that ECEC can lease resources that meet the requirements according to the change of user request rate,and effectively reduce the request delay and system overhead through cooperation.