A structured lighting approach to image analysis for robotic applications using camera-space manipulation

This dissertation describes and illustrates with experimental applications a versatile and robust means for determining imaging information as required for autonomous robot control using the method of camera-space manipulation (CSM). In typical industrial settings robots are controlled in a “teach-repeat” mode of operation. In order to overcome the limitations of this method researchers combined vision sensors with robots in an effort to emulate human hand-eye coordination. However, in this process several new difficulties arose including image analysis and the implementation of visual information in a control scheme. Despite much research in image analysis, existing algorithms are limited due to the need “to infer the state of the physical world from the inherently noisy and ambiguous images of the world.” [57] The implementation of visual information in a control scheme is equally challenging. A popular robotic vision strategy, called calibration, uses the camera(s) as a kind of measuring device to locate the physical coordinates of the workpiece. Calibration, however, is inaccurate globally and therefore restricted to limited regions in highly structured environments. Another common problem of calibration is its brittle nature; calibrated camera parameters must be updated frequently. This dissertation addresses both the control strategy and image analysis through the application of CSM and structured lighting to a large class of real-world problems. CSM derives its accuracy and robustness by carrying out the control of the manipulator entirely within the two-dimensional image plane of the participating, widely-spaced cameras by establishing a relationship between the appearance of visual features on the manipulator and the internal joint configuration of the robot. The structured lighting device, a multiple-spot laser projector, establishes compatible targets within the reference frame of each participant camera as required for CSM through a process of “matching” the image-plane appearance of each laser spot in each participant camera. Whereas conventional image analysis techniques must make geometrical inferences from noisy image data in this “spot-matching” process the surface geometry of the workpiece is “captured” directly within the laser spot data, even while the CSM requirement of defining camera-space targets that are consistent with the same physical-surface junctures is met.