Research On The Parameters Extracting Of Hand Gesture Based On Stereo Vision

Virtual Reality (VR) technology, as an advanced form of human computer interaction techniques, can make people immerse in virtual environments and have a sense of immersive. In virtual reality, hand gesture interaction is a senior human computer interaction. Making the motion parameters of gesture input to the virtual environment is a key technology to achieve the gesture interaction. At present, there are mainly two ways in realizing the gesture input: one is based on the data glove; the other is based on the computer stereo vision. With the development of computer vision, the gesture input, based on the stereo vision, can make hand movement get rid of the shackles of the data glove, and is becoming the research focus in the future. Extracting the gesture motion parameters, especially the joint angles, is the key to achieve the gesture input.The extraction of the gesture motion parameters, which is based on the computer stereo vision, is mainly studied in this article, and the main work and innovation are as follows:1. In this paper, the physiological structure of human hand and the principle of hand movement are studied, and then the physiological structure is simplified. According to the simplified model of human hand, a virtual hand model is established, which is very close to real human hand and can be controlled by motion parameters.2. A method of marking the feature points of the gesture joints based on color and shape combination to reconstruct the 3D coordinates is presented, and 23 joint points are marked. Two hand gesture images are obtained with one camera in the same zoom from two positions. The threshold method is used for image segmentation to extract the feature points,which color and shape are all different, then these feature points are matched by the same mark. The Zhang Zhengyou's method is used to calibrate the intrinsic parameters with plane calibration template by two steps, and then the different camera projection matrixes, which are formed when the two images are being shot, are computed. Using these matrixes, the 3D coordinates of the joint feature points are reconstructed.3. An algorithm based on the 3D coordinates of the 23 joint feature points is presented to extract the gesture motion parameters and the formulas are deduced to compute these parameters. The hand joint feature points'coordinates, based on the world coordinate system, are transformed to the coordinates based on the wrist coordinate system. According to the inverse kinematics principle, the formulas, which are used to compute the joints angles of the thumb and other fingers, are deduced. And motion parameters of a specific gesture are solved. Then these angles are input to the virtual hand model to simulate the real hand gesture and a virtual reality experiment system is achieved, in which virtual hand can be droved by the real hand gesture with the computer stereo vision.4. In order to verify the effectiveness of the proposed method in this paper, many experiments, in which different hand gestures were obtained under different experiment conditions, have been implemented and the experimental results have been analyzed. The results of experiments show that the real hand gestures are visually and livingly simulated by virtual hands, and it is proved that the methods used in this paper are correct and feasible. Expected purpose is reached.