Cloud Manufacturing Service Composition Based On Deep Reinforcement Learning

Cloud manufacturing is a service-oriented manufacturing model based on emerging technologies such as cloud computing,Internet of Things,and big data.It integrates distributed manufacturing resources and conducts centralized management to provide consumers with on-demand manufacturing services,thereby promoting comprehensive resource sharing and improving resource utilization efficiency.An important issue in cloud manufacturing is to combine value-added services composed of multiple different manufacturing services to meet the multistage complex manufacturing needs of users,that is,the problem of service composition.The use of heuristic and meta-heuristic algorithms in the service composition that determines the processing order of subtasks will not only reduce the flexibility of the algorithm,but also cause problems such as manual adjustment of algorithm parameters when the scale of the task or service changes and the service is unavailable.Therefore,the paper introduces deep reinforcement learning(DRL)algorithms to solve the single composite task-oriented manufacturing cloud service composition problem.Deep reinforcement learning algorithm is an artificial intelligence algorithm that combines deep learning's ability to perceive complex environments and reinforcement learning's ability to make decisions on problems.The algorithm learns the inherent patterns and rules between different service combinations to find an effective solution to the problem of cloud manufacturing service composition.Deep reinforcement learning algorithms are divided into two types: algorithms based on discrete variables and algorithms based on continuous variables.In the cloud manufacturing combination problem,not only the optimal service corresponding to the current subtask needs to be considered,but also the overall quality of service(Qo S)for the combined result of the task needs to be considered in the case of meeting the Qo S constraints of the maximize task requirements.Therefore,the paper introduces the Deep Deterministic Policy Gradient(DDPG)algorithm based on continuous variables to solve the service composition problem in the cloud manufacturing environment.First,a cloud manufacturing service combination model that considers the quality of logistics services is proposed,including a single composite task model,a resource model,and a logistics service model,combined with corresponding combination indicators.Secondly,the objective function and constraint conditions of the model are given,and Markov Deecision Progress(MDP)is established based on the model.Finally,the DDPG algorithm is used to solve the cloud manufacturing service composition problem under the simulation case.In order to verify the effectiveness of the deep deterministic policy gradient algorithm and the ability to adapt to complex manufacturing tasks and the large amount of uncertainty in the manufacturing process under the cloud manufacturing environment,this paper sequentially carried out the effectiveness,adaptability and robustness of the algorithm.In order to verify the performance of the algorithm intuitively,this article uses the ant colony algorithm as a benchmark and compares it with the Deep Q-Network(DQN)algorithm to verify the performance of the algorithm used in this article in all aspects.The experimental results show that the deep deterministic policy gradient algorithm can effectively solve the problem of large-scale cloud manufacturing service composition.At the same time,in a dynamic manufacturing environment,when the service is unavailable,the algorithm proposed in the paper has better adaptability than the value-based DQN algorithm.Finally,the task set and resource set are expanded and experimentally verified.The results show that the use of DDPG algorithm can effectively solve the problem of manufacturing cloud service composition to single composite tasks in a dynamically changing environment,and the use of ant colony algorithm to solve this kind of problem When there is a problem,it not only requires manual parameter adjustment and cannot be effectively expanded,but the model trained through the algorithm proposed in the paper is more suitable for solving the service composition problem in the cloud manufacturing environment.