The Research On Self-Adaptation Technology Based On Reinforcement Learning

Self-adaptive system is typically situated in some environment and should adjust its structure and behavior according to the changes of external environment and internal state, in order to improve the flexibility, robustness and friendliness of system, and satisfy the design objectives in a better way. Recently, as more and more applications are deployed in Internet environment, self-adaptive systems are widely used in many domains like enterprise computing, industry control, amusement, etc. How to develop such complex systems has become a hot topic in the literature of software enginnering.The existing approach to developing self-adaptive systems adopts some language to pre-define the environment changes and self-adaptive behaviors at design-time. However, changes in many environments like Internet are unpredictable and uncertain. Therefore how to adapt to the possible changes of environment become difficult and impossible based on current method.The thesis investigates the self-adaptation technologies based on software agent and reinforcement learning. We believe that the researches on self-adaptation technonogies in open and uncertain environment should alter research approach: (1) self-adaptation decision should be made at run-time (not design-time). (2) self-adaptation decision should be made by software system (not designer). Based on the above method, the thesis conducts researches on the self-adaptation running mechanism, implementation method, supported tools, etc.Firstly, reinforcement learning is introduced into the research of self-adaptation to solve the problem of uncertain and unpredictable changes of environment and the related issues such as self-adaptation decision at run-tme. Mechanism and algorithm of self-adaptation based on reinforcement learning are proposed. Second, we believe autonomy is the basis of self-adaptation. Software agent technology is helpful to achieve self-adaptation decision according to the environment changes. The architecture of self-adaptive system is proposed by extending agent model. Third, the development framework is proposed and software tool is developed in order to support the development and construction of self-adaptive systems. Lastly, a case is studied and implemented to illustrate the feasibility and effectiveness of the proposed approach.