Deep Reinforcement Learning Based Researches On Caching Strategy And Data Transmission In Heterogeneous Network

With the commercial use of 5G and the advent of the short video era,the traffic demand is growing explosively.The problems of spectrum shortage,link congestion,channel interference and so on are becoming increasingly serious,and the wireless communication network is under great pressure.However,at the same time,a large number of homogeneous contents are transmitted repeatedly in the network,resulting in a great waste of resources.Ease network congestion and increase the transmitting speed have become an urgent problem.Mobile Edge Caching(MEC)can significantly reduce the backhaul congestion of wireless communication system and greatly reduce the traffic pressure of the core network.Moreover,by unloading the homogeneous content to the edge of the network,it can also significantly reduce the user access delay and improve the system traffic transmission rate,so as to greatly improve the overall system performance.As one of the important supports of 5G technology,the application of massive multiple input multiple output(MIMO)technology has greatly improved the user edge communication rate.The main work and research results of this work are as follows:(1)The optimization of mobile edge caching strategy in heterogeneous networks is studied.Firstly,a hierarchical heterogeneous network communication system is constructed.Based on the energy loss of base station and transmission energy loss between users,the optimization problem of minimizing system energy consumption is established with MEC technology.The original problem is transformed into an unconstrained minimization optimization problem by using the indicator function and the vectorized representation of the cache decision.The transformed problem is non convex,so this paper proposes a cache decision optimization scheme based on reinforcement learning.On the user side,the Q-learning algorithm is used to make the local optimal cache decision based on the greedy idea,and on the small base station(SBS)side,the deep Q network(DQN)algorithm is used to make the global optimal cache decision within the coverage.Simulation results show that the proposed scheme can effectively identify the heat file and formulate the cache strategy.(2)The optimization of data transmission in heterogeneous networks is studied.By introducing large-scale antenna array and MIMO communication,this paper adopts a two-stage hybrid precoding scheme and optimizes the hybrid precoder as a result of compromise between performance and hardware cost.This chapter mainly considers the point-to-point downlink transmission process between SBS and users,and establishes the optimization problem with the goal of maximizing spectral efficiency.The main channel information is obtained by singular value decomposition method,the problem is transformed into the optimization problem of chord distance minimization between the optimal encoder and the two-stage coding scheme.Owing to the non convexity of the problem,this chapter proposes a deep reinforcement learning algorithm based on DDPG to solve the digital precoder.Finally,this paper explores the performance of the algorithm by constructing different reward functions.Simulation results verify the generalization performance and effectiveness of the proposed algorithm.