Gesture Interaction In Monocular Augmented Reality

With the rapid development of computer technology, some novel HCI (human-computer interaction technologies which fit the interpersonal communication habits become very active including face recognition, expression recognition, lip reading, head motion tracking, gaze tracking, hand gesture recognition, pose recognition and so on. The center of HCI technology has gradually transferred from the computer to the human himself, HCI technology becomes an interactive technology with both multimode and multimedia.For mobile AR(augmented reality) equipment, especially the wearable AR equipment, the traditional HCI means (such as mouse, keyboard and touch screen) cannot fit the demands of natural interaction due to the lack of portability. Hand gesture is a very natural, intuitive, vivid interaction mode, it can complete the interaction with the virtual objects vividly and intuitively by imitating traditional input operation (click or drag). The main target of this article is to fulfill the interaction between the bare hand and virtual objects in monocular augmented reality.There have been many gesture-based interaction systems in AR, but the problem of real-time dynamic gesture tracking under complex scenes still has not been solved properly especially for the mobile AR environment. This article was mainly about gesture recognition under complex scenes, we improved the updating method of commonly used dynamic skin color model and proposed a method to construct the skin color model rapidly based on two adjacent frames which could greatly expand the application situations that use vision-based gesture interaction. We improved the hand recognition methods based on fingertip detection and2D hand model and attempted a hand recognition method based on Machine Learning, we also implemented an interaction system in AR.