The integration of material models and computer vision for force and displacement sensing

The manipulation of deformable objects is an important problem in robotics and arises in many applications including biomanipulation, microassembly, and robotic surgery. In microrobotics and space robotics the robotic manipulator itself is often deformable. This dissertation discusses the use of computer vision to provide feedback for robotic interaction with deformable objects and to provide feedback when the robotic manipulator itself is deformable. Computer vision is a logical sensing choice for working with deformable objects because of its wide availability across many fields and the richness of the data provided by a vision system. A template based deformable object tracking algorithm will be introduced that can be used to provide force and displacement feedback for robotic applications. Various material models are used for modeling the template deformation including the beam equation, the boundary element method, and neural network models. It is important that the tracking algorithm used for feedback is robust to occlusions and spurious edges in the source image. Approaches to handle these potential difficulties are presented. Finally, a compliant, four degree of freedom, MEMS microgripper is presented. Because of this gripper's compliant design, vision tracking can be used to provide position and force feedback to greatly simplify its design and fabrication. Vision-based force and displacement sensing is capable of providing accurate and robust feedback where other sensing techniques are not possible or are difficult to implement.