Research On Adaptive Service Composition Using Multi-Agent Reinforcement Learning

In service computing, as the runtime environment are changing and services online are evolving over time, web service composition is faced with performance challenges by the dynamic and complex environment all the time. As a consequence, the methods for service composition must achieve self-adaptability to tackle the problems of uninformed behavior evolution of services and changes of the outside environment, so as to maintain their performance. In addition, such methods should also maintain high efficiency in the large-scale scenarios for service composition, which is necessary when applied for practical applications.This thesis introduces a new model for adaptive service composition based on multi-agent reinforcemen-t learning in dynamic and complex scenarios. On basis of reinforcement learning, the Multi-Agent System and the fictitious play process in game theory are integrated into the model proposed in this thesis. Among them, the reinforcement learning technique is used to handle the problem of adaptability in a highly-dynamic environment, the Multi-Agent System applied in this model is aimed at improving the computational efficien-cy and the fictitious play process in game theory is used here to enable agents to work for a common task. Moreover, on the base of this model, we propose an algorithm based on off-policy reinforcement learning and an algorithm based on on-policy reinforcement learning for adaptive service composition, and proved their convergence theoretically.Finally, a series of simulation experiments are conducted in this thesis to verify the effectiveness, scala-bility and self-adaptivity of our approach.