Study On Mapping Of Visual Gesture Interaction

Due to the advantages inclueding low attachment, weak intrusive and without usage of devices, distant freehand or gestural techniques based on computer vision are regarded as the most natural interface, and slowly becoming a hot topic in Human-computer Interaction(HCI). Although distant freehand techniques can provide more instinctive user experience than traditional methods, such kind of methods still face the usability problems including instability and low precision because of the openness of mapping, hindering the popularization of freehand interface.Mapping in HCI refer to the spatial relation and response mode between Control Space and Display Space. Different from traditional methods, mapping of distant freehand interaction is one kind of many-to-many relationship in dynamic environment instead of a fixed one-toone relationship among human, task and device in static scenes, as user is not restricted in the space near the deivce and cannot have complete control over the terminal by getting the device. To improve the naturalness of freehand interaction, implement a human-centered interaction and bring human-computer interaction near to a status of human-human interaction, it is proposed to probe the physical and social characteristics of mapping of freehand interaction, take a deep insight into the complexity of HCI and study user’s behaviors in the perception environment of frehand interaction.Based on the research and development of visual gesture interaction techniques, methods combining qualitative analysis with quantitative measurement were used to study mapping characteristic of freehand interaction and user’s behaviors in such kind of environments and the achievements are as followed:(1) Study on user’s mapping mode preference and comfortable control space. Mapping mode of gesture interaction determines how computer responds to the action of user’s hand motion. Comfortable hand motion range is an important guideline for a user-centered design of mapping of gesture interaction. Various kind of mapping can be designed due to the variety of geture. The mapping charaterics of freehand interaction was analyzed from the views of interaction technology, rgonomics and cognitive psychology. A method was proposed to study user’s mapping mode preference by learning user’s hand movement control according to the interface context in actual interaction scene. On the other hand, user’s comfortable control space was study according to relationship between user’s performance and hand motion range.(2) Study on human pointing precision in freehand space. Pointing precision reflects user’s capability in finishing precise interaction tasks. A user-centered definition of pointing precision was proposed and its connotation was expounded from the views of motor neurology and information technology. A method based on Fitts’ Law was proposed to measure human pointing prection in three seperate dimesions when using freehand techniques, contructing the relationship between pointing precision and dimensions(horizontal, certical and depth) as well as task amplitude.(3) Study on variability of interaction space and its effect on user’s behavior in freehand interaction. The reasons for variability of interaction space in freehand interaction were analyzed according to the development of freehand techniques and its application scenes. According to the analysis, a method was proposed to study the influence of variability of interaction space on user’s visual perception of visual space and effects of user’s locations relative to display space on user’s object performance, learning the relationship between user’s performance and the difference between the effective and actual control space.(4) Research and develop a user-centered mapping for freehand interaction. A usercentered mapping was proposed according to the research results mentioned above. This mapping techniqeus could adjust the CD gain according to user’s behavior, reducing the probability of large amplitude hand movement and keeping user’s performance stable.The studies of this dissertation not only took deep insights of the complexity of HCI, which provided effective theoretical directions for user-centered design of freehand techniques, but also would promote the research level in HCI of China in several aspcests including theoretical discussion, research methodology and experimental approach. In addition, work of this dissertation could help to learn about user’s behavior when interacting in open space as freehand interaction is one kind of methods for open interaction space. This would also provide useful references for freehand-like interaction techniques development in smart space such as smart house, digital home.