DNN Inference Business Scheduling System Based On Deep Reinforcement Learning

In the past few years,DNN has proven to be a universal and effective tool for solv-ing many practical problems.The number of applications using DNN has exploded.The DNN inference business has begun to become one of the services provided by the cloud computing environment,and low-latency,high-accuracy response quality has become the goal.Due to the good realization of deep reinforcement learning in the AI field,peo-ple have also tried to use deep reinforcement learning methods to achieve high-quality response of DNN inference business systems.This article mainly studies the following issues:1.DNN business scheduling based on deep reinforcement learning.Today,most of these problems are solved by carefully designed heuristics.A careful study of the latest research in this area can reveal that the typical design process is:(1)Ingenious heuristics are proposed to simplify the problem model;(2)Heuristics are carefully tested and ad-justed to achieve good performance in practice.If certain aspects of the problem(such as workload or interest metrics)change,this process must usually be repeated.The deep neural network of deep reinforcement learning has strong feature extraction capabilities,without artificially building models,just input observations to the system,and formulate goals,reinforcement learning agents will learn the ideal strategy in the direction of the goal.Even if some aspects change,reinforcement learning does not need to be remod-eled.This article uses DQN,Double DQN,Dueling DQN,AC,A3C and other common deep reinforcement learning to solve the scheduling problem of DNN inference business.The response quality of the DNN request is a function expressed by the processing delay and the accuracy of the result.The range is between 0-1.The results show that all methods can achieve a request response quality of more than 0.9 on the test data.2.Online learning problem of DNN inference business scheduling based on deep reinforcement learning.In the research of question 1,it is found that when the reinforcement learning agent whose training has converged is applied to some new environments that are very different from the training environment,because these features are not learned during training,the re-sults during testing are not good.However,in actual applications,it is normal when the environment is constantly changing,so agents must learn online learning.This paper first studies the online learning of A3C and DQN,based on the task is continuous in a short time,using short-term small batch learning,the agent uses the previous short-term experi-ence to train,and then applies to the subsequent tasks.Although this method can achieve a very high request response quality,the training takes a long time.Then this article uses a model-based meta-learning method,which directly models the dynamic system.The model is very sensitive to changes in parameters,and a small amount of training data can be used to quickly converge.The results show that although the request response quality is not as good as the former,the model is very stable and the use time is greatly reduced.Previous work has not distinguished users,nor considered the fairness of users.In this pa-per,user tags are added to consider user fairness.This article first considers user fairness on the server side,and still uses the deep reinforcement learning method,and adds the user fairness indicator to the goal of the agent,but the result is not ideal,because this method will produce a lot of useless experience,it is worth The relatively little learning experience makes it difficult for the agent to learn.So this article considers this problem and the next problem together;4.Multi-agent learning based on deep reinforcement learning.Move the agent from the server to the client to study the multi-agent learning problem.More and more mobile phones have begun to have machine learning capabilities,and machine learning terminalization has become a trend,so this article considers placing agents on the client side,and also solves problem 3 in the multi-agent learning method.This article uses multiple independent AC agents to learn independently,because in reality,each user can only get their own information is reasonable.Each agent enters a state that only it can see and outputs its own decision,but each agent's goal is to maximize the quality of its own request response and maximize the fairness indicators for all users.The results show that this method can not only ensure the quality of the user's request response,but also ensure the fairness between the users.