Research On Path Planning In Migrating Workflow Based On Reinforcement Learning

With the development of computer and internet technology, the research of the migrating workflow management systems is being paid more and more attention by people. In this model, migrating instance (MI) is considered well placed as the executive agent, using local resources and services provided by a working position to executing one or many tasks. When MI finds the current working position can not meet the requirements of its execution, MI carrying the task specification and the execution result migrate to next another position that satisfies its requirements and continue executing. The choice of the next position depends on the target and the requirement of the current task. Therefore, how to find a best path becomes one of the hot issues in the research of migrating workflow, and this is also the focus of this paper.According to the ideas of the migrating workflow, the process of the execution of MI can be understood as the process of state transitions in the migrating workflow, namely, starting from a given initial state, MI constantly migrates between the working positions that provide services, then produces a new state, Until the new state meets the requirements of MI’s goals. So path planning for workflow is also the strategy selection of state transition. Reinforcement learning is is a goal-oriented learning. Through these interact continually, the learners selecting actions and the environment responding to those actions and presenting new situations to learners, which includes four elements: states, actions, reward functions and environment model. By combining working path planning for MI with reinforcement learning, this paper establishs a model of path planning based on reinforcement learning according to environment information. This model can easily measure differences between the states of migrating instance and an evaluation is given by using the theory of reinforcement learning, then choose the best strategy of state transition, namely, migration path.At first, this paper puts forward, with social acquaintance network as the environment network of migrating instance, a static path planning method based on Q-learning in the migrating workflow. This method can plan out a global migration path, by recycling the signal of environmental feedback and the information state of current task.In consideration of the complexity, flexibility and uncertainty of network environment, it is very difficult to plan out a global migrating path for MI to adapt to the dynamic changes in the environment of working network. Concerning this issue, this paper further proposes a dynamic path planning method based on κ-steps Q learning in the migrating workflow. The method finds a next best working position from the current position by using the future κ-steps information of the migration decision. while executing tasks, at the same time, it dynamically plans a path-workingng for migrating instance.At last, this paper designs simulation experiments under the three groups of network environment that the social member nodes is respectively500,1000and2000, and the numerical results are compared and analyzed. Finally we find by experiment, these two methods can solve the problem of path planning for MI:the first method can single obtain a complete, reliable, efficient migration path, that has strong feasibility and high efficiency, but inadaptation of dynamic changes of environment; the second method that be applied to path planning for MI, has strong flexibility. Therefore migrating instance can plan a path dynamically, especially that has good adaptability and performance in the dynamic and complex environment.