Complete Pure Hand Gesture Vision Based Human-Computer Interaction Technology

Computer vision based human-computer interaction is one of the most important topics in computer science community. It employs the analysis of vision information to directly understand human action, and achieves natural and barrier-free user experience, thus bears great research and market merits. Recently, the hand gesture based visual interaction attracts more and more attention. However, comprehensive and applicable real-time systems on general computing platforms are yet in deficiency. This dissertation employs intangible bare hand gesture input and integrates novel gesture detection, tracking and recognition alternatives to facilitate system-level solution on general computing platforms. Specifically, the contribution of this dissertation is as follows:(1) Hand gesture detection. The proposed hand gesture detector utilizes off-line trained AdaBoost gesture detector and mixtures of Gaussian model based skin color blob detector. Combined with confidence information from specific gesture recognizer and multi-frame historical records, a highly reliable specific gesture detector is fused on Fischer criterion, which improves the robustness of the interactive hand model against background clutter and camouflage colors compared with mono-detectors.(2) Hand tracking. For the desktop interaction scenario, the dissertation improves the extensively deployed kernel-based tracker with a discriminative feature subspace descriptor of high confidence. By constructing mixture of Gaussian kernel function based similarity encoding spatial and color feature information, target scale adaptive multiple kernels estimates the discriminative similarity surface around true target region, thus the mean-shift iteration is calibrated to adapt deformable and fast hand motion against background clutter in a fast and precise way. For the far distance interaction scenarios, the particle resample mechanism is improved with mixture of Gaussian kernel function based similarity; the particle propagation strategy is promoted with a target motion adaptive state transition model. Thus the particle filter tracker adaptively locates small target in outburst motion mode. (3) Hand gesture recognition. To address the ambiguity between tracking and recognition, the cell-wise histogram of oriented gradients hand posture descriptor and the hierarchical nearest-neighborhood classifier with the soft-decision strategy are proposed to achieve fast recognition of multiple hand postures from real-time tracking results. Furthermore, the design regulation for a user-friendly real-time interaction system is discussed to employ hand dynamic gestures combining trajectory and postures.(4) Hand gesture interaction system. The dissertation discusses the hardware requirements and information processes for visual hand gesture interaction. Further, a hand gesture interaction system for the automotive intelligent center is proposed. It incorporates the aforemented detection, tracking and recognition module seamlessly, designs the lexicon field in detail and realized the software prototype.Consequently, the dissertation provides theoretical algorithm analysis and experiments, which not only set forth the advantages on efficiency and accurancy for all of the detection, tracking and recognition ingradients, but also outperform the current related literatures on completeness and uniqueness.