The LAP-based Control Of A Multi-finger Mechanical Dexterous Gripper

Using free- hand gestures for remote control of objects an effective interaction way because the hand gestures are natural forms of communication and are easy to learn. A single gesture can be used to specify both a command and its parameters indicating the positions and movements of the hand and fingers, which provide a higher potential of expression. Using free hand as an input device eliminates the need for employing intermediate transducers.The ways for recognizing a hand gesture were classified as sensors-based and vision-based. Sensors-based ways use sensors physically attached to the hand to recognize the gesture, one of them is the "Data Glove". The mechanical data glove linked to the computer has the shortcoming of spoiling moving comfort and thus reduces the autonomy; in addition, the "glove" is expensive and could be damaged easily, the number of order gestures is also limited. Vision-based ways use optical sensors to detect the object, image processing and analyzing technologies were combined to perform the gesture recognition. The study about hand gesture analysis in vision-based system is mainly focused on the explanation of sign language. The research deals with steady positions recognition instead of dynamic gestures because a system that interprets gestures and translates them into a sequence of commands must have a way of segmenting the continuous stream of captured motion into discrete lexical entities, this process is somewhat artificial and necessarily approximate.The research introduced in this paper used "looking at people"(LAP), a new application area of computer vision in recent years, in the action control of a dexterous gripper with 3 fingers and 9 DOF. Instead of data glove, the system uses a pair of CCD camera and image processing and analyzing technologies for detecting the hand gesture of the operator.The camera pair observes the hand gesture of the operator from different directions and a pair of gesture images is caught. Image processing, image analyzing are performed to segment the key points on the hand and get the correlation of the points, stereo-vision theory is used to calculated the coordinates of the key points in the 3D space. The comparison between the physiological structure of the person hand and the mechanical structure of the dexterous gripper was performed to find a suitable way for transforming the information of the hand gesture to an executable parameter for the dexterous gripper.This is a subject-crossed research concerned with robotics, computer stereo vision, image processing and analyzing technologies. The main works done in the research are the following:A set of robust algorithms for detecting the key points positions on the hand and their co-relations was designed. It is suitable for the non-rigid structure with some small movement such as human's hand. The algorithm could search all the key points on the hand under arbitrary orientation and rearrange them according to their host finger and their position on the fingers.On the basis of Lumigraph, the new theory in computer vision, an algorithm forcalculating the coordinates of the key points above in 3D space was developed. So the solid geometric calculations could be employed in the calculation of 3D information without calibrating the camera and the calculating the projection matrix, a complex matrix calculation process. Lining the space points up can restructure the hand gesture in 3D space.The research uses the task-based modeling in the control of the dexterous manipulator instead of command gesture. The operator observes the position and orientation of the object and design the grasping gesture for the mechanical gripper, shows it with his own hand. As introduced above, the system calculates the gesture information and the fingertip information of the operator was mapped as the fingertip position of the dexterous gripper. The inverse motion equation derived from the motion equation could be used to calculate the joints' angles on 3 fingers.The vision-based grasping control of the dexterous gripper suggested in the paper provides the dexterous gripper with the direct grasping gesture instead of the order sequence in the data glove system, the fingertip mapping algorithm makes the gripper have precise fingertip position in the grasping action and can control the grasping for the objects with uncommon surface. Getting rid of a glove linked with computer makes the operator more free and comfortable. Any action of the hand beyond the monitoring area of the CCD will not affect the controlled device. The number of cameras in the system is not limited and the cameras in the system could be arbitrarily grouped as pairs without considering the relative position between them. This method was considered quite applicable for a multi-camera system, it could resolve the occlusion problem occurred in human-machine communication. This method is expected to be used in other research works, such as modeling a virtual hand in a virtual circumstance, tracking the motion of a human limb and fast 3D reconstruction of an object.