Deep Reinforcement Learning Based Coded Caching Strategy In Fog Radio Access Network

Fog Radio Access Network(F-RAN)is a solution to 5G Radio Access Network(RAN)which has lower fronthaul cost,lower transmission delay,and lower network congestion.It provides more communication information and control function.Deep Reinforcement Learning(DRL)has both distinguishing feature of Deep Learning(DL)and Reinforcement Learning(RL),which makes it able to solve complex problems with both perception problems and decision problems.For F-RAN in the scene of file popularity changes,the collaborative coded caching and caching replacement process can be modeled as a single agent perception decision problem in discrete-time system(DTS)when the system state space and action space are determined.The state transition action is determined by the reward function,and combined with the DRL method,the local optimal solution for the decision will be given.The thesis mainly focuses on DRL methods and DRL heuristic algorithms in the artificial intelligence aera,such as DB3C and CBA3C algorithms,which solve the problem of cooperative coded caching strategy and coded caching placement problem in F-RAN under the scene of file popularity changes.It mainly includes the following two parts of research content and conclusion:For the F-RAN collaborative coded caching strategy problem in the file popularity change scenario,we used the DQN-based heuristic algorithm named DB3C to construct a DRL model that satisfied the problem scenario and determined the system state space and the action space as well as the reward function according to the jointly successful transmission probability,which achieved the convergence of the algorithm in the file popularity changing scenario.Simulations compared the performance of the DB3C,RL algorithm,MPC based caching algorithm with coding,and MPC based caching algorithm without coding on the system reward,which proved that DB3C has better performance.In response to the problem of F-RAN cache placement problems under the file popularity change scenario,we used the heuristic algorithm named CBA3C that combined the Camul algorithm and the A3C multi-threaded training mode to construct a DRL model that satisfied the problem scenario,and determined the system state space and action space,and comprehensively considered the cache hit rate and system cost to derive the reward function,which realized the rapid convergence of the algorithm in the file popularity changing scenario.Simulations respectively compared CBA3C,LRU,LFU,MARKING and RL algorithms to prove that the CBA3C heuristic algorithm could guarantee the system better cache hit rate while achieving lower total system cost.