Realtime And Robust Hand Tracking From Depth Research

Hand tracking is important in many human computer interaction applications and has beenintensively studied for decades. However, it remains challenging due to its great complexities: thehand is highly articulated with complex finger interactions; it moves fast with large viewpointvariations.In spite of significant progress in recent years, the state of the art approaches are limited incertain aspects. Some realtime and robust systems are limited in recognizing discrete handgestures only, supporting a small number of DOFs, or under a fixed viewpoint. Those limitationsare due to difficult tradeoffs between the system complexity and targeted goals. To achieve highaccuracy and robustness, previous works use complex models, sophisticated cost functions andexpensive optimization.We present a realtime and robust hand tracking system using a depth sensor. It tracks a fullyarticulated hand under large viewpoints in realtime (25FPS on a desktop without using a GPU)and with high accuracy (below10mm).To our knowledge, it is the first system that achieves suchrobustness, accuracy, and performance simultaneously.Our system is made of several novel and effective components. We use a simple hand modeland define a fast cost function. Those are critical for realtime performance. Previous optimizationmethods are not suitable for our simple cost function. We instead propose a hybrid optimizationscheme that overcomes their drawbacks, achieves quick convergence and good accuracy. Wepresent new finger detection and hand initialization methods that greatly enhance the robustnessof tracking.