Adaptive visual servoing in uncalibrated environments

The calibration accuracy is one of major factors that affect the visual servo control performance. Generally, in order to achieve high performance visual servoing, an accurate calibration of vision system must be made with substantial efforts and time. To avoid the tedious and difficult calibration work, in this thesis, two major classes of visual servo control for robot manipulators in uncalibrated environments have been developed.; First, we address the design of the position-based visual feedback controllers for set-point control and trajectory tracking of a manipulator when the homogeneous transformation matrix between the manipulator base frame and the vision frame is not calibrated. It is assumed that the intrinsic parameters of the camera have been calibrated accurately and an efficient algorithm is available to estimate the pose (position and orientation) of the end-effector of the manipulator. Based on an important observation that the visual Jacobian matrix can be represented as a product of a known matrix, which depends on the kinematics of the manipulator, and the unknown rotation matrix R between the manipulator base frame and the vision frame, we propose an adaptive algorithm to estimate the unknown rotation matrix R on-line. In the proposed controllers, the position errors measured in the vision frame are fed back to the manipulator through the estimated Jacobian matrice. The estimated rotation matrix is convergent to the true value when the desired velocity is persistently exciting (PE).; Second, we extend the first method to feature-based (image-based) visual servoing. In this approach, the two dimensional information of a set of image features on computer image plane are selected and fed back to the controller via a composite image Jacobian matrix, in which the intrinsic parameters of camera, the transformational matrix between the planar manipulator frame and the camera frame and the depth between the camera and manipulator are all not calibrated and measured. Based on the important observation of the composite image Jacobian matrix and the perspective projection model of camera, an adaptive law is designed to estimate the mixed and unknown parameters including the unknown intrinsic parameters, extrinsic parameters and the depth of the planar manipulator-vision system on-line, then a method is expounded for on-line composite image Jacobian estimation that realizes uncalibrated feature-based visual servo control.; In both approaches, dynamic effects of the manipulator are considered in the controller design. It is proved with nonlinear dynamics of the robot by Lyapunov approach that they both guarantee asymptotic convergence of motion errors in the general and strictest senses. The performances of the controllers have been also demonstrated by extensive computer simulations and experiments on a 3 DOF robotic arm. The simulations and experimental results both confirmed expected behavior of the controllers. The use of the proposed visual servo controllers would greatly simplify the implementation of a manipulator-vision workcell. The controllers are especially useful when a pre-calibration is impossible, such as when a manipulator works with an active vision system carried by a mobile robot.