Machine vision in the real-time control of a robot manipulator

The objective of this dissertation is to investigate the use of machine vision as an active visual servo as well as a 3D sensor in the real-time robot control.; First the machine vision as a 3D sensor in the robot control is studied. A new method is developed to compute 3D object position. An over-determined linear equation is developed. It computes 3D object position more accurately than the traditional depth equations. This method eliminates alignment procedure required in traditional parallel and convergent stereo systems. It also detects and quantifies misalignment in the parallel and convergent stereo vision system. With use of this method, not only the setup procedure of a stereo vision system can be simplified greatly, but also a high degree of accuracy can be achieved. The simulation and experiments have proven that an accuracy of one hundredth of an inch can be easily achieved.; An active visual servo scheme is proposed. A new full-state feedback control law is developed to regulate the visual error to zero, thus forcing the robot positioning errors to be compensated dynamically. The full state feedback gain matrix is chosen to ensure the vision system stability, controllability, and observability. Unlike the traditional "look" and "move" machine vision applications where the vision system is used as a 3D sensor and a path planner, the proposed scheme uses the vision system as an active control element as well as a position sensor. Simulation has demonstrated that the visual servo technique is feasible and effective for robot error correction in both positioning and dynamic path following control tasks.; Finally a real-time visual servo is implemented by utilizing maximum hardware processing capacities and parallel software structure. By employing double image buffers, overlapping the image processing and image transfer, and executing finite state machine and parallel image processing techniques, the visual servo is able to operate continuously at 5 Hz without violating the hard real-time constraint. The experiments are conducted on an accurate CNC machine and the redundant gantry Kraft robot. The results have demonstrated the stability and convergence of the visual servo technique and its promising prospect for future practical industrial applications.