Research And Application Of Gesture Recognition Technology Based On Head Mounted Camera

Human-computer interaction technology grows rapidly with the development of the computer science. Gesture recognition technology and augmented reality technology have been widely studied in recent years, and become important development topics of human-computer interaction technology. This paper discusses the difficulty of the interference of human faces in hand gesture recognition. Hence we consider fixing the camera on the back of the hand, so that the camera's view is consistent with the eye view of the user, which will not affect the interaction between human and the computer.This paper focuses on the gesture recognition technology based on head mounted camera. Hand gesture recognition technology mainly includes gesture segmentation, fingertip detection and static gesture recognition. Gesture segmentation is based on dynamic skin color algorithm in YCrCb color space and the improved frame difference algorithm. The combination of the two methods realizes a good gesture segmentation results in real-time video and is more robust to similar skin color objects. The fingertip detection algorithm combines the polygon fitting, curvature and centroid distance method. In addition, we assume some criterions to identify fingertip points according to the characteristics of the hand. Experimental results show that the average accuracy of fingertip detection was 92.4%. In the static hand gesture recognition, we define three effective hand shapes, which are index finger, fist and palm. And through the combination among those three hand shape, the gesture commands are converted to controlling signals of the mouse and keyboard, such as cursor move, click, double click, right click, zoom in, zoom out and keyboard operations. And the average success rate of gesture manipulation was 92.57%.In the combination of the above gesture recognition methods and the augmented reality technology, a system is proposed which can interact with virtual characters by gestures in real time. This system can control the virtual characters to appear, disappear, zoom in, zoom out, move and dance. Experimental results indicated that the system achieved a satisfactory human-computer interaction and the average accuracy of each gesture interactive instructions in real-time video reached more than 90%.