Research On Reinforcement Learning Based Dynamic Cache Allocation For Cloud Computing

The cache interference phenomena in cloud computing seriously decreases the efficiencies of the applications and increases the response latency and service cost of the system.Traditional allocation strategies usually optimize the instant performance of the applications such as the miss rate and instructions per cycle based on the statistical analysis,which cannot optimize the long-term efficiencies of the applications aim at the change of running state such as the data accesses.Therefore,this thesis focuses on the cache interference problem and investigates the dynamic cache allocation technique for multi-applications in cloud computing to evenly improve the utilization of the cache,efficiencies of the applications and the quality of service and user experience of cloud computing.First of all,a deep reinforcement learning based cache allocation strategy is proposed in the thesis.Value-based and Actor-Critic reinforcement learning structures are applied for designing cache allocation strategies according to the features and suitability of complete isolation and partially overlapping allocation pattern respectively to determine the allocation scheme and optimize the performance such as the miss rates of the applications without knowing the running characteristics.Experiments are performed to evaluate the strategies.In the experiment in which 5 applications contend for the cache with 12 ways,the proposed allocation strategy reduces the average miss rate of the applications by 13.2%and improves the average instructions per cycle by 14.2%,compared with the benchmark strategy in complete isolation pattern,and further reduces the average miss rate of the applications by 5.3%and improves the average instructions per cycle by 10.4%in the partially overlapping pattern.In the thesis,we propose a safe reinforcement learning based cache allocation strategy,which applies the self-adaptive stability measurement regarding the dynamic means and standard deviations to improve the stability against imperfect exploration and optimize objective,reduce the visits to risky states and actions and increase the learning speed.The results of the experiment based on 5 applications contend for the cache with 12 ways show that with more computation consumption and decision time,the proposed allocation strategy can decrease the standard deviations in partially overlapping and complete isolation pattern by 75.9%and 90.6%,respectively,compared with the deep reinforcement learning based cache allocation strategy.