Edge Server Video Caching Algorithm Based On DQN

Thanks to the fast growing of user population and video population in recent years,Internet traffic has been heavily dominated by online video streaming.Nevertheless,it gives rise to the challenge to meet the stringent quality-of-service(QoS)requirement of video streaming with limited bandwidth resource.Caching videos on edge servers in the close proximity of users is an effective approach to reduce backbone traffic and improve video streaming performance.Yet,there are two challenges which have not been well-solved by the existing works.First,video popularity is so dynamic that even the popular videos may only last for a few hours.Secondly,the replacement cost has not been considered appropriately and comprehensively.To solve this problem,we propose a real-time caching framework for edge network to simulate the real-time caching of edge server.In this caching framework,we consider the replacement cost and interrupt cost of the edge server when replacing the video,so that the edge server can perform cache updates in a very short period of time.On this basis,we use the deep reinforcement learning to solve the problem of video caching on edge servers.We utilize the Exploration-Exploitation capabilities of deep reinforcement learning algorithms to address the tradeoff between the long-term and short-term benefits of cache updates.However,due to the limited computing power on the edge server,the algorithm based on deep learning has been unable to run effectively.Using the original deep reinforcement learning algorithm to solve the cache problem will result in an exponential increase in the size of the algorithm's state space and decision space as the number of videos increases.Obviously,this contradicts the limited computing power of edge servers.we propose to solve the video caching problem on edge servers by leveraging the Deep Reinforcement Learning(DRL)framework.We design a novel lightload video caching algorithm named as DQN based Online Video Caching(DQN-OVC).We change the prediction of all video decision earnings to that of a single video decision earnings.On this basis,the original search for the optimal decision in the exponential cache space is changed to a single traversal of the single video earnings to obtain the optimal strategy.Our improvements greatly reduce the computational complexity of the algorithm and enable our algorithm to execute periodically in a short period of time so that it can be updated according to the latest video trends.In addition,it also enables our algorithm to adapt to different sizes of input,so that the edge server can save a lot of computing resources when the number of users is small.Finally,we use real Internet video user access records for simulation experiments.Experimental results show that in terms of hit ratio,our algorithm has an improvement of 10% to 30% compared with traditional cache algorithm and prediction algorithm based on deep learning,and can reach the theoretical optimal hit ratio of 93% at peak period.