Research On Technology Of Computer Vision-based Virtual Hand Interaction

As the appearance of the virtual reality (VR) technology, new styles of human-computer interaction (HCI) also have emerged as the times require. Traditional 2D HCI is developing towards 3D HCI. The VR HCI is different from the former HCI. This new-style HCI should incarnate harmoniousness, naturalness and human-centered features. Especially, tracking the human-hand and recognizing its gestures have played an important part in technology of HCI. It has been researched in order to implement convenient and natural communication mode.Aiming at the features of Multi-Orientation Virtual Environment (MOVE) system in VCC division, this dissertation takes hand tracking and recognition into human-computer interface. It facilitates the development of the system and provides a wider application perspective. The research contents include:1) Application of binocular-vision technology in HCI: Aiming at solving the faults of Data-Glove that is expensive and not flexible for user, this dissertation proposes a method that the movement of hand is tracked by camera, which reduces the requirement for hardware and avoids the disadvantages brought by Data-Glove. It is easier for binocular-vision technology to gain space positions of objects. This dissertation focuses on static gesture recognition based on binocular-vision technology. The recognition process is simplified by setting key-points on fingertips, which also assures the real-time requirement of recognition.2) Research on hand gesture simulation algorithm and its implementation: Human hand structure contains high degree of freedom, and its gestures have more diversity and more space differences, which also increase the complexity for hand gesture simulation. Based on analysis of hand kinetics model and summarization of hand movement constraints, space positions of fingers are calculated by joint kinetics simulation algorithm on the basis of simplified hand model for computation, and hand simulation results are displayed in the virtual environment. Corresponding experiments have been performed to prove the validity of proposed algorithms.3) Hand gesture simulation based on skeletal animation: In order to achieve much more realistic simulation result, the principle of skeletal animation is introduced in this dissertation. Mapping the more realistic hand model to itscorresponding skeleton model realizes that movement of hand model is driven by the movement of skeleton model. The work lays the foundations of the natural interaction in MOVE system in the future.