Research On Bare Hand Interactive Method Based On Depth Fusion In Augmented Reality

Currently,the application of augmented reality(AR)system is increasingly widespread.In an AR system,camera-tracking,virtual-real fusion and human-computer interaction are the three core issues.When the user uses AR system,bare hand interaction is one of the most natural interactive modes.There are still many places to improve in the AR system in hand interactive scenarios.For instance,only a small amount of depth cues are intro-duced,the degree of fusion is poor.At the same time,the interactive mode is limited in two-dimendional image space or gesture recognition.In response to these issues of human-computer interaction and virtual-real fusion in AR system,an AR framework is proposed which is based on multi-channel bare hand real-time motion information.The information includes:the hand positon and posture obtained from the external hand information gathering device,the flexible virtual hand interactive model,the hand depth map got by rendering the virtual hand model and the skin color mask gotten by skin color detection.The method of acquiring the information and how to apply it in virtual-real fusion and human-computer interaction are described detailedly in this paper.In the stage of virtual-real fusion,the overlay of the real hand and the virtual objects is implemented based on the hand depth information and hand skin color;casting shadow from real hand to virtual hand is implemented based on the model of the virtual hand and virtual objects.When interact with the virtual object,you can choose either of recognizing the hand gesture by processing location information,also controlling the virtual objects by collision detection.In this way,you can achieve the goal of natural human-computer interaction.