Hand Gesture Interface System Based On Near Infrared Computer Vision

With the development of computer science technologies and wide use of projectors in human's daily life, Human Computer Interaction (HCI) based on projectors is becoming more and more popular. Compared with traditional input devices, such as keyboards and mice, hand gesture is a more natural, intuitive and meaningful way for people to communicate with computers. Moreover, hand gesture based HCI interface using computer vision techniques is low-cost and easy to deploy.The vision-based HCI interface is natural because the user's hand gestures are captured by cameras in a touch-free manner. However, vision-based methods are sensitive to changes of background, illumination, etc. In the case where both projector and camera use visible light, the lights from projector typically deteriorate the images with complex background, resulting in low-quality captured hand shape, which significantly further complicates processing on the images and prohibits real-time performance. By contrast, near-infrared-based devices utilize lights within different spectrum; therefore do not suffer from such interference from projectors. In addition, the background in captured image is less complex and analysis on the image is less complicated. Real-time performance is therefore more feasible. In this paper, we analyze the projector-camera based HCI system, and design near-infrared-based hardware. Also, we study the image processing algorithms that are related to our system, such as moving target detection and hand gesture analysis. Our contributions include the followings:1. Modularization of the projector-camera based HCI system. Our system consists of the following six components: image acquisition module, geometry calibration module, hand area segmentation module, hand gesture analysis module, hand gesture trajectory recognition module and HCI interface module. Besides, the functionality of each module and interfaces between modules are defined.2. Design of the near-infrared-based hardware. The choice of near infrared is based on analysis of properties of images captured by various video acquisition devices based on different light frequencies. We propose a solution that combines active and passive near-infrared sensor; design a near-infrared LED array light source for the active sensor.3. A moving target detecting algorithm based on adaptive update window. Through comparisons of several algorithms for moving target detection and studies of properties of hand movements in our system, we propose a new moving target detection algorithm based on adaptive update window. Our approach can perform fast segmentation of a hand from the background in a near-infrared image.4. An optimized Neural Gas algorithm based on the topological representation of hand shape. We discuss the neural network algorithms based on self-organizing feature maps, and their application on the topological representation of hand shape. We optimize the original algorithm as well as its computational procedure according to the properties of hand shape. With the proposed algorithm, the topological representation of hand shape can be successfully generated from a segmented area of the hand within a short computational time.