Web Services Composition Optimization Based On Reinforcement Learning

Web service which is a kind of component on the Internet shows good encapsulation, loosely coupled and cross-platform. However, the single Web service has inevitable limitations in functionality, it can't provide people with more function and complex service. In order to reduce the cost and time of developing, and achieve services value-added and reuse, we need to combine existing services to form a combination of services which can satisfy user's demand. There are a lot of services with the same functions in the process of service composition, one of the key issues is to select service by using QoS.The dissertation researches on optimal methods of Web services composition , which is based on Reinforcement Learning. On the one hand, the dissertation analyses the existing Web service composition methods, and studies the QoS properties of Web service. On these basis, the dissertation presents the model of QoS, and proposes the QoS calculating method of a single service and composition services. Without considering dynamic nature of Internet environment and stochastic of Web service, existing methods mostly generate static plans, as a result, Web service composition often fails with larger probability. Web service composition(including sequence and parallel structure) is modeled by using Markov Decision Process(MDP), then Q learning is used to obtain the optimal service composition policy. The results show higher success rate of service composition by using dynamic controlling method.On the other hand, response time is a significant factor in the process of Web service composition, it is affected by network load and service itself. It is modeled by using finite state continuous time Semi-Markov Decision Process(SMDP), then Q learning algorithm is used to obtain the optimal service composition policy. At last, the simulation is used to illustrate the feasibility and effectiveness of the algorithm.