Learning by imitation and exploration: Bayesian models and applications in humanoid robotics

Learning by imitation is an important mechanism for rapid acquisition of new skills in humans and robots. A critical requirement for learning by imitation is the ability to reason under uncertainty. Uncertainty arises during the process of observing the teacher as well as from the imitator's own dynamics and interactions with the environment. This dissertation introduces new probabilistic methods for learning novel, generalizable behaviors in a humanoid robot via imitation. At the heart of my approach is a proposed algorithm for selecting actions based on probabilistic inference in a learned Bayesian network. This inference-based action selection technique affords an efficient, straightforward method of exploiting rich, yet uncertain, sensory data gathered by the robot.; Many existing methods for planning robotic actions require an engineer to explicitly model the complex physics of the robot and its environment. This process can be costly, tedious, error-prone, brittle, and inflexible to changes in the environment or the robot. The method I propose involves learning a predictive model of the robot's dynamics, represented directly in terms of sensor measurements, solely from exploration. Experiments are performed with a Fujitsu HOAP-2 25-degrees-of-freedom humanoid robot and the Webots dynamic simulation software. I present results demonstrating that the robot can learn dynamically stable, full-body imitative motions simply by observing a human demonstrator and performing explorative learning. Additional results show how the inference-based action selection technique can be used for policy learning, where sensory feedback can be used to adapt behavior online. I present policy learning results for a lifting behavior (learned via imitation) that generalizes to a wide range of objects of novel, unknown density. Besides imitation-based learning, this dissertation makes other contributions to the emerging area of robotic learning. First, intractability due to very high-dimensional state and control spaces is tackled using dimensionality reduction techniques. Second, nonparametric techniques are introduced to handle the problem of learning and inference with continuous-valued random variables.; Ultimately, this thesis seeks to contribute novel ideas which one day may form the basis for a powerful human-robot interface which allows people to quickly and effortlessly train robots to perform new skills.