Research On Expression Of Gesture Style For Three-Dimensional User Interface

Progress in the field of human-computer interaction (HCI) has developed new technologies that allow users interact with computer in more natural and comfortable ways. Human-computer interaction can be capable of more direct, harmonious and convenient communication between people and computer. Natural human-computer interaction can make users interact directly with computer by capturing the habits of behavior or the model of operation formed in their daily lives, which stressed that it requires no training or special training, thus more closely similar to human-human communication. Hand plays an important role in human life and is a significant way for people to communicate with the outside world. Gesture is an important input interface in the human-computer interaction system, which can express users’ intention. Compared with the traditional way, gesture interaction can enable users to get rid of the mouse and keyboard, and has great value and broad applications.Implementation of three-dimensional gesture interaction based on the gesture animation library is an important way to achieve rapid and robust human-computer interaction. The gesture retrieved from the animation library may simply mean that it is consistent with the actual gesture semantics and may be differences of the motion styles including the direction, speed etc. How to obtain the mathematical transformation applied to the animated gesture and make the transformed gesture animation match with the users motion styles is the core problem to be solved in this thesis. The essence of the problem is how to describe the users mental model, which has strong scientific value; It may provide a different user experience for smart television, desktop application, mobile phone and other gesture interface systems, which shows broad application value. This thesis is supported by the National Natural Science Foundation of China (61472163 and 61173079) and the Key Project of Natural Science Foundation of Shandong Province (ZR2011FZ003). The main study and innovation are as follows:(1) The gesture interactive platform based on data glove and monocular visionThe gesture interactive platform based on data glove by data glove and position tracker can obtain real hand motion data and then drive the three dimensional virtual gesture realizing the conversion between virtual gesture and real hand, which is an efficient method of capturing motion data. The gesture interactive platform based on monocular vision can get the hand image by camera and then segment the original image, which lays the foundation for further analysis.(2) Guided by the cognitive theory, to establish the gesture behavioral modelIn the certain interactive tasks, the hand motion has strong regularity. From the perspective of cognitive psychology, this thesis combine the gesture interactive platform based on data glove and monocular vision, and get the real hand motion data and image information at the same time. The law hand motion and the process of state are analyzed and summarized to study the mathematical relation between the degree of freedom and hand image. This thesis establishes the gesture behavioral model from the three aspects of speed, trajectory and perspective to construct multiple gesture interactive styles.(3) The gesture style method based on behavioral modelGesture with different stages of the hand motion has different styles. During the translational stage, the main styles contain trajectory, perspective and translational speed. During the grab and release stage, the main styles contain grab and release speed, respectively. The gesture style method based on behavioral model is applied to express the mental model in natural interaction interface, which can reduce the cognitive load and improve the use experience. The main contribution of this algorithm is that we establish mathematical relation between the actual hand and gesture animation. This mathematical description not only is possible to construct multiple gesture interactive styles, but also provides a new method for the expression of the users’ mental model.