Gesture-based Human-computer Interaction And Fingertip Detection Algorithm

The traditional human-computer interaction is confined to the applications of keyboards, mouses and joysticks for a long time, whose naturalness and friendlines have already been unable to meet people's requirements. Vision-based gestural interaction allows the computer to interact with the users by recognizing the gestures and finger motion trajectories. A common camera is needed to obtain image sequences of the finger movement and extract features of the hand. The techniques of bare-hand HCI include image processing, machine vision and pattern recognition and other related disciplines, which is at present a research hot spot. Moreover, human computer interaction based on gestures has quite broad applications in sign language recognition, game control, intelligent home and etc.Fingertip detection and recognition is the key of the finger-based interaction system. A lot of domestic and foreign scholars have proposed many approaches for fingertip detection for several yeras, but almost no algorithm can adapt to any of the scenes. Some factors such as the dynamic diversity of background and the random of finger movements, making finger's precise positioning still a very challenging task.Fingertip detection belongs to moving object detection. Both motion information and color clues are considered to be quite important features and uesd widely. This article studies the object detection techniques based on skin color, background extraction and image differencing, and then compares some fingertip detection algorithms in both theory and experiment. Background substraction is one of the simplest way, whose result largely depends on how to update the background model. Image differencing can adapt to dynamic environment, however, it will fail when the object moves slowly or rest in the scene. Skin color is another useful imformation as it differs from other objects obviously, however, it is very sensitive to illumination.On this basis, this paper considers fingertip's shape for detection, including the finger's regional property (template) and its contour analysis. The two algorithms are closely related with the result of object segmentation as both of them are operated after segmenting. Region-based approach defines a fingertip template to detect the fingertip by template matching, making it more sensitive to the hole in foreground region; Contour-based method determine the location of your fingertips by comparing the value of curvature. It will judge the junction of two fingers as fingertips when the threshold is improperly set.This paper presents a simple fingertip detection approach based on a modified Radial Symmetry Transform (RST). And the fingertip is finally located by skin color discrimination. The algorithm can be directly operated on the original image, which will not be affected by the poor segmentation. Our modified radial symmetry transform greatly enhances the robustness to illumination change, and can be real-time. Experimental results show that this method for fingertip detection is feasible and effective.