Self-improvement through self-understanding: Model-based reflection for agent adaptation

The ability to adapt is a key characteristic of intelligence. This dissertation explores techniques for enabling intelligent software agents to adapt themselves as their functional requirements change incrementally. In the domain of manufacturing, for example, a software agent designed to, disassemble physical artifacts may be given a new goal of assembling artifacts. As another example, in the internet domain, a software agent designed to browse some types of documents may be called upon to browse a document of another type.; This research examines the use of reflection (an agent's knowledge and reasoning about itself) to accomplish adaptation (incremental revision of an agent's capabilities). Reflection in this work is enabled by a language called TMKL (Task-Method-Knowledge Language) that supports modeling of an agent's composition and functioning. A TMKL model of an agent explicitly represents the tasks the agent addresses, the methods it applies, and the knowledge it uses. TMKL models are hierarchical, i.e., they represents tasks, methods and shell called REM (Reflective Evolutionary Mind). REM provides support for the execution and adaptation of agents which contain TMKL models of themselves.; This dissertation presents a variety of strategies for adapting agents. Some of these strategies are purely model-based: knowledge of composition and functioning directly enables adaptation. This model-based approach is combined with two traditional artificial intelligence and machine learning techniques: generative planning and reinforcement learning. The combination of model-based adaptation, generative planning, and reinforcement learning constitutes a mechanism for reflective agent adaptation which is capable of addressing a variety of problems to which none of these individual approaches alone is suited. The work described in this dissertation has demonstrated the computational feasibility of this mechanism using experiments involving a variety of intelligent software agents in a variety of domains.