Research On Natural Gesture Recognition In Interactive System Based On Projector-Camera

Human computer interaction system refers to the system that directly supports an interactive communication between human and computer, which is mainly used to achieve the information transmission to improve efficiency and friendliness of computer systems. With the rapid development of computer technology, human-computer interaction technology advances. To establish natural, harmony, human, intelligent human-computer interface has become a core objective of the new generation of human-computer interaction research. This paper involves the projector-camera (Procam) system, in which a projector is used as output device, input device camera as a new type of interactive multimedia platform. The system creates dynamic interactive fantasy environment through the integration of advanced projection display technology, computer vision, pattern recognition technology and artificial intelligence technology, where gesture is a new model of human-computer interaction and achieves more natural interaction. Human-computer interaction system based on Procam exists many problems about the current research and development, such as extreme projected light by projector, serious interference to gesture detection and location brought by complex illumination. The variable forms of manpower, uniform surface color, less texture information, difficult features extrication method and recognition algorithm are hard to meet the real-time operational requirements, which leads to difficulties to achieve the system. In this paper, for the above technical problems different from other interaction systems, the method of segmentation gestures is proposed with illumination compensation and elliptical skin model, and multi-feature combination to achieve gesture recognition process by stepwise design method.First, calibrate the Procam system. By analyzing the imaging principle of the camera, and considering the first order radial distortion and tangential distortion, to realize cameras calibration based on Open CV checkerboard. Establish a mapping between the two models, and finally geometrically calibrate Procam system using projective transform algorithm, which provides the foundation for the subsequent process of the system.Second, achieve bare hands detection under the complex projection light. Firstly, in order to achieve the accuracy of subsequent skin color segmentation, illumination compensation approach is used to reach gesture image color cast correction. And then color segmentation techniques are used to segment gesture area. The robustness and real-time of the different methods by comparing the experimental results are verified.Then, extract and recognize gesture features. Step by step, we judge and elect effective gestures in accordance with the idea of recognition of gestures. According to the calculation amount of feature extraction, gestures are recognized from small to large. The calculation of other features could be cut down, if the gesture features are enough to determine, which improve efficiency; while if a single feature is insufficient to ensure the reliability of the judgment, combination of other features are needed to achieve the judgment.Finally, achieve the whole system. Experimental verification of the proposed method is made with hardware and software built.