Research On Hands-free Manipulation Of Virtual Objects For Human-Computer Interaction

Hand movement and it's gesture is a kind of natural and direct mode of communication in our life. As human became the center of the Human-Computer Interaction (HCI), hand gesture recognition which based on graphic recognition became a focus in this field. Because the gesture recognition is based on the plane characteristics, so it can only send the input information based on the plane projection of the hand, this constrain the operation in the 3D environment and limit the users' input information.Augmented Reality (AR) is the further development of Virtual Reality (VR), the target of an AR system is to emerge the virtual objects with the real environment. The existing of virtual objects mixed with real objects can strengthen the user's feeling and interaction to the real world. 3D registration is based on the real object's characteristic matching, and using the special marker as the matching objects has became the most mature and most stable method. Thus, tie the marker to the hand, and use the 3D registration technology to recognize the 3D position, we can achieve a better 3D control by using the hand directly.In this paper, we first analyses the development and application area in Human-Computer Interaction based on AR technology. With the comparison between the physic sensor based hand input technique and hand gesture recognition input technique, we came to conclude that it's the best way to combine the marker registration technique to the hand gesture recognition which based on the graphic recognition.We forward a complete and feasible hand capture virtual objects system framework which based on the AR environment. The construction of the hand capture virtual objects system can be divided into three main parts: Building the AR registration environment, Hand capture operation recognition, Management of the virtual objects.Grounded on the analysis of hand recognition and hand capture virtual object rules, we put some efforts to design the key modules of the computer graphic. The research content include: Definition of hand gesture, 3D hand capture rule, hand capture recognition, characteristic recognition of hand photo, the transformation between the hand and marker.With the camera-glass integrative HMD as input/output device, we build the completed hardware platform and software framework. Via the hand capture virtual objects experiment based on the AR environment, the feasible of our HCI technique is proved.