A Magnetic Hand Tracking System

Recent advances in miniaturization of computers and widespread applications of mobile computing have made it possible for people to wear a computer and to perform numerous functions within the personal space. However, this important application requires a human interface system that is both more compact and more functional than those are available today.The human hand, as one of the most dexterous organs, is potentially a natural human-computer interaction "device" in its own right. It is able to produce numerous postures due to its large number of mechanical degrees of freedom and its massive connections to the neural system. If the human hand can be tracked by a simple, unobstructive apparatus, it will be able to deliver information to the computer by finger motions and hand postures in a free space. This will allow more rapid and convenient communication with portable and mobile systems and enable more ubiquitous computer applications in people's daily lives. In this dissertation, we present a novel magnetic hand tracking system, which is wireless, portable, unobstructive and convenient. With specifically designed software that captures hand postures, this system can be used to convey information to the computer and support a variety of man-made systems that require human-machine interaction. The main contributions of this dissertation are as follows:1. A magnetic hand motion tracking system is originally proposed. In this system, small permanent magnets and contactless magnetic sensors are used to track finger motion. A magnet, which can be integrated with an artificial nail, is affixed to each fingernail to mark the motion of the finger. When fingers move, the combined magnetic fields produced by the magnets at fingertips are recorded by a set of magnetic sensors around an electronic wristband. The recorded data from the sensors are utilized to inversely calculate the hand posture. Both the forward and inverse models of the system are provided from which a two-step hand tracking scheme is proposed. The first step of the scheme estimates the locations and orientations of the fingertips from the measured magnetic flux densities, and the second step reconstructs the hand posture from the locations and orientations of the fingertips.2. Algorithms for the location and orientation of the fingertip(s) are proposed. First, a dipole model of the magnet marker is investigated and the properties of this model are studied. Then, the dipole model is applied to the localization of fingertips using a least squares (LS) dipole fitting method. Finally, a linear algorithm, which requires only simple matrix and algebraic operations, is studied and exploited for the localization of a single fingertip. Both the LS dipole fitting algorithm and the linear algorithm are evaluated by computer simulations. Our results show that, as long as the number of measurement channels is no less than that of the fingertips' location and orientation parameters, the LS dipole fitting method is able to localize fingertips accurately using noiseless measurements. Our evaluation on the noise properties of the system shows that, although the LS dipole fitting method degrades with the increase in the number of fingertips to be localized, this effect can be compensated by increasing the number of magnetic sensors. With noiseless sensor signals as the input and using five three-axis sensors, we were able to localize and orientate single fingertip successfully using the linear algorithm. In noisy environments, however, a denser sensor arrangement must be adopted in order to achieve higher localization accuracy and overcome the sensor sensitivity to noise. Fortunately, the increase in the number of sensors does not usually impose a practical constraint.3. The posture reconstructions of the index, middle, ring, and little fingers are investigated. First, a geometric model of the index, middle, ring, or little finger is established consistent with the hand anatomy. Based on this geometric model, we demonstrate that the posture of the index, middle, ring or little finger is locally and uniquely determined by the location and orientation of its fingertip except for four special cases which are either unnatural in finger motion or rarely performed. Thus, this result confirms a high solvability of our method in hand posture reconstruction involving all fingers except the thumb. In order to calculate hand posture, we utilize an LS method with a local optimization feature to solve the inverse problem (using locations and orientations of the fingertips to computer hand posture). The effectiveness of this method is demonstrated by simulations.A six-sensor hardware prototype is developed, which consists of the following functional units:(1) sensor output amplification, (2) sensor noise handling, (3) environmental magnetic interference cancellation, and (4) acquisition and transmission of measurement data. Experiments based on real world data were conducted using a prototype to evaluate the hand tracking system. The basic feasibility of our system was demonstrated in terms of measurability of hand motions. The feasibilities of the magnet model and the hand model were validated by comparing actual measurements of the magnetic data and model-based simulation results.In summary, a new hand tracking system featured with a simple, inexpensive and convenient structural design has been developed. An LS dipole fitting method has been utilized to localize and orientate fingertips. An LS hand posture reconstruction algorithm is constructed to estimate the joint angles of the hand. Our computer simulations and experiments using real-world data have both shown the effectiveness of our new system.