Research Of Key Algorithms On ProCams Based HCI

The traditional Human Computer Interaction research was restricted to hardware devices such as keyboard, mouse and control rod, which have been used for decades and can hardly meet people's demand on convenience, comfortableness, usability and so on. The HCI system based on computer vision has come to be the disquisitive highlight in recent years. Vision based HCI involves several research areas, including computer vision, image processing, pattern recognition and AI, and has a extensive application prospect in natural HCI, hand gesture recognition, virtual reality, smart home system, body sense game and so on. Compared with traditional HCI system, HCI in projector-camera system has several advantages as follows: the hardware devices are simple and easy to gain, the user experience is very good, more interactive, enhances virtual environment's reality and immersion. Several difficult technical problems have to be solved while building a projector-camera based HCI system, such as the modeling of photometry system in ProCam, hand segmentation in complex illumination environment and the detecting of fingertip in ProCam system. These fundamental yet challenging questions are of universal applicability and transplantablility.This thesis systemically explores various key technologies in depth during the design of ProCam based HCI system. The main work and innovation in this thesis are presented as follows:1. The photometric model in ProCam system is carefully studied. Based on the photometric model, a simple and efficient photometric calibration method is realized in this thesis. Besides the photometric calibration, this thesis introduces perspective transform technology to solve the geometric calibration problem. Former geometric calibration methods only take into consideration the distortion of cameras, while neglecting the effect of lenses in projectors. By adopting perspective transform, the new calibration process can absorb the nonlinear effects of the projectors. Meanwhile, RANSAC based algorithm is introduced to calculate the perspective transform matrix, which helps further eliminate tolerances.2. Presents a bare hand segmentation method based on skin reflecting characteristics. On the basis of the photometric model of ProCam system and the peculiar human skin reflective property, the presented method can segment hand or other human skin area under dynamic and extremely complex illumination environment including the projected light by projectors. 3. Radial Symmetry Transform (RST) based finger detection algorithm is improved in this thesis. The prior method, which utilize skin color for kicking out of candidate target points, would easily affected by light condition or semicircular area of finger joints. The presented improvement method in this thesis excludes negative points by analyzing shape features of finger. Experiments prove that this algorithm is feasible and effective.4. A set of ProCam based HCI system is explored. Compared with current projector-camera based HCI systems, the hardware structure of this system is much simpler with only one ordinary projector and single CCD camera. No extra markers or gloves are needed during the interaction. Tests show that the system has good availability and practicability.