Service Composition Via Automatic Hierarchical Reinforcement Learning

In the field of service computing, developing efficient solutions to achieve automatic service composition has drawn significant attentions. With the continuous development of Web service technology,the internal and external environment of services changes dynamically,there are more and more candidate services with same function, and the composite business process becomes more and more complex. These factors make developing service composition with adaptability and high efficiency become research hotspots.Reinforcement Learning is a kind of machine learning method that explores the optimal policy by interacting with the outside world, it has been used in service composition to achieve self-adaptability. However, faced with service composition problems with larger and larger scale,because conventional reinforcement learning doesn't have a relatively fast convergence rate, it can hardly satisfy the service composition's demand for high efficiency.In light of the above problems,this thesis proposes a service composition approach via automatic hierarchical reinforcement learning technique, the main research contents include four aspects. Firstly, a service composition modeling approach based on Semi-Markov Decision Process is proposed to satisfy the precondition using automatic hierarchical reinforcement learning technique to solve the service composition problems. Secondly, the optimal policy is defined combined with the service composition model based on Semi-Markov Decision Process.Thirdly, the paper proposes an self-adaptive service composition algorithm via automatic hierarchy reinforcement learning,the algorithm contains two parts: first step is the hierarchical division of the service composition task via automatic hierarchical reinforcement learning,in place of manual hierarchy each time when encountering new scenario,so as to improve the efficiency of hierarchical division and save time, second step is the service composition optimization based on MAXQ hierarchical reinforcement learning. Fourthly, a series of experiments are conducted to assess the effectiveness of the proposed approach. Experimental results show that the proposed service composition optimization algorithm have good validity,scalability and adaptability. When encountering large-scale service composition scenarios, it can converge very fast with great efficiency.