Towards a learning system for robot hand-eye coordination

Through careful consideration of the Hand-Eye Coordination (HEC) problem, it can be viewed as the process of performing a sequence of transformations from an input space to an output space. Specifically, the entire process from eye to hand can be viewed as a mapping from scene space to arm configuration space. This single mapping may be broken into a sequence of mappings from one space to another. The sequence we have chosen to model is the following: scene space to image space; image space to camera coordinate system; camera coordinate system to arm/world coordinate system; arm/world coordinate system to arm configuration space. Additionally, an active vision system is incorporated which introduces an image space to head configuration space mapping.; Given the view that these subtasks are mappings from a N-dimensional input space to a M-dimensional output space, this research presents a unified framework by which the various subtasks of the HEC problem may be implemented. This framework uses a recursive partitioning algorithm to build a hierarchical tree classifier which uses a nearest neighbor classification based on the Voronoi tessellation as its decision making criteria. The resulting data structure is a Recursive Partition Tree (RPT), which is the heart of the framework. The topology of the RPT is not determined a priori, or hand-coded. Instead the topology is allowed to develop during its construction, based on the given set of training samples and the order in which they are presented to the construction algorithm. Each node of the RPT represents a cell of the space which is further partitioned by its children via a Voronoi tessellation. Each leaf node corresponds to a training sample and stores the corresponding output. This general framework provides us with a method for systematically dealing with the complex relationship between the sensors and the manipulator. In the performance phase, given an input, the RPT is used to retrieve the desired output. The RPT results in a logarithmic average time complexity in the number of stored training samples.; Extensive simulations have been performed with two implemented modules. Experiments using a real setup demonstrate the ability of a system using RPTs to accomplish the stereo calibration, head-to-object space mapping, and point-to-point movement of the hand within the HEC task. Experiments using real data required that the current approach be simplified since collecting real data for training proved to be much more difficult and time consuming than generating simulated data. Nevertheless, promising results are obtained.