Research On Content Delivery With Deep Reinforcement Learning In Data Centers

In the era of big data,the content delivery service of Content-centric Data Center Networks(CCDCNs)is expected to offer user the better satisfaction of Quality-of-Experience(Qo E)than that in the conventional networks.Nevertheless,the satisfaction of Qo E becomes the major challenge in CCDCNs along with the growth of big data.Therefore,to improve the satisfaction of Qo E in this paper,we introduce the content delivery mechanisms.Based on the proposed big data architecture,we first design a Tensor-Fast Convolutional Neural Network(TF-CNN)algorithm for extracting the hot contents while improving the training speed.On the basis of hot contents,we then introduce the caching method to decision-making intelligently.Finally,we mainly focus on the cache allocation in CCDCNs.The main contributions in this paper can be summarized as follows:(1)We propose a novel big data architecture consisting of three planes in CCDCNs.Specifically,the data storage plane stores a wide variety of data collected by sensors and originated from different data sources.Then,the data processing plane filters,analyzes and processes the data to make decisions autonomously for extracting high quality of contents.Finally,the application plane initiates the execution of the events corresponding to the decisions delivered from the data processing plane.Under this architecture,we particularly use TF-CNN algorithm based on Deep Learning(DL)to balance the amount of hot contents and training speed of CNN.(2)We intend to study the content-centric caching in CCDCNs.As requirements are stochastic,we use Reinforcement Learning(RL)architecture to jointly determine the Q-value.Estimating the Q-value can be conducted in the Deep Neural Network(DNN)since the states and action spaces are in a large scale.Unfortunately,training DNN models can lead to RL instability.To address this issue,fixed target network,experience replay and adaptive learning rate are proposed to balance the Q-value accuracy and accelerated stability in DRL.(3)On the basis of the Shortest Path Tree(SPT)and node centrality,we devise two suboptimal dynamic algorithms,which are suitable for CCDCNs with cache allocation frequently.Then,based on the DRL,we devise a cache allocation algorithm with converge to near-optimal solution.