Research On Big Data Management Based On Deep Reinforcement Learning In The Edge Environment

In the era of big data,the big data services of edge environments are expected to offer end-user the better satisfaction of Quality-of-Experience(QoE)than that in conventional environments.Nevertheless,the satisfaction of QoE becomes the major challenge in edge environments along with the increasing of big data.Therefore,aiming to improve the satisfaction of QoE in this thesis.On the one hand,we first design QoE models including the QoE model with two factors of big data accurancy and transmission rate.On the other hand,we design QoE models including the QoE model with two factors of caching latency and caching cost.Then,we introduce the big data management algorithm.Based on the proposed big data architecture,we design a Tensor-Fast Convolutional Neural Network(TF-CNN)algorithm.Considering the transmission rates of hot contents can affect the satisfaction of QoE,we then apply the discrete method based on mathematics to the transmission rate-segmenation.As a result,the transmission rate-segmenation can infinitely close to continuous.As the increasing of hot contents,the edge environments suffer from the pressure of high traffic throughout.Therefore,we finally introduce the caching method in edge environments to decision-making intelligently.The main contributions in this thesis can be summarized as follows:(1)To improve the satisfaction of QoE,we first design the QoE model including two factors of big data accurancy and transmission rate.We seek to balance the big data accurancy and transmission rate.On the other hand,we design another QoE model including two factors of caching latency and caching cost,inorder to get the trade-off between caching latency and caching cost.(2)Then,we design a novel big data architecture consisting of three moudles in edge environments.Specifically,the data storage moudle stores a wide variety of data collected by sensors and originated from different data sources.Then,the data processing moudle filters,analyzes and processes the data to make decisions autonomously for extracting high quality of hot contents.Finally,the application moudle initiates the execution of the events corresponding to the decisions delivered from the data processing moudle.Under this architecture,we particularly use TF-CNN algorithm based on Deep Reinforcement Learning(DRL)to balance the amount of hot contents and training speed of CNN.Considering the transmission rate of hot contents,we introduce the discrete method based on mathematics.The method can infinitely segment the transmission rate of discontinuous data to achieve continuity.Simulation results show that our proposed big data architecture and big data mangagement algorithm can obtain a higher QoE than others.To improve the satisfaction of QoE,we design a novel big data architecture consisting of three moudles in edge environments.Specifically,the data storage moudle stores a wide variety of data collected by sensors and originated from different data sources.Then,the data processing moudle filters,analyzes and processes the data to make decisions autonomously for extracting high quality of hot contents.Finally,the application moudle initiates the execution of the events corresponding to the decisions delivered from the data processing moudle.Under this architecture,we particularly use TF-CNN algorithm based on Deep Reinforcement Learning(DRL)to balance the amount of hot contents and training speed of CNN.Considering the transmission rate of hot contents,we introduce the discrete method based on mathematics.The method can infinitely segment the transmission rate of discontinuous data to achieve continuity.Simulation results show that our proposed big data architecture and big data mangagement algorithm can obtain a higher QoE than others.(3)Since the increasing of hot contents and traffic throughout pressure of edge servers,we intend to study the edge caching in edge environments.We first introduce the caching architecture in this environemts on the purpose of QoS.Then the caching requirements are stochastic,we use Reinforcement Learning(RL)architecture to jointly determine the Q-value on the basis of the caching architecture.Estimating the Q-value can be conducted in the Deep Neural Network(DNN)since the states and action spaces are in a large scale.Unfortunately,training DNN models can lead to RL instability.To address this issue,fixed target network,experience replay and adaptive learning rate are proposed to balance the Q-value accuracy and stability of DRL.Finally,simulation results show that our proposed caching framework and content caching scheme can obtain a higher QoE than others.