Research On Fingertip Tracking And Locating Technology Based On Binocular Vision

The light Flight Simulator is equipped with Head Mount Display (HMD) to display virtural scene. Compared with traditional flight simulators, Virtual reality based light flight simulators, i.e. VR simulators, have great potentialities in the applications for compact structure and relatively low price. However, the pilot with HMD does not see the real environment inside the cockpit,which needs to be generated in VR scene. Moreover, it is difficult to interact between human and machine.This dissertation researched several key problems in the vision-based recognition and applied this technique into the developmet of a new kind of vision-based flight training helmet device.One of the key problem in the development of vison-based helmet is the precisely fingertips tracking and locating of trainee. Based on binocular stereo vision, this dissertation innovatively researched vision-based hand tracking and fingertip recognition in simple background. Firstly, a new method based on skin abstraction combining with motion information is proposed by this dissertation in the stage of hand detection. The HSV color space and Gaussion filtering method on skin color modeling are introduced to abstract the skin color. Then by using the image subtraction operation the skin-color like background and face area can be eliminated. The algorithm can make the gesture division in complex background accurate. Then in the part of tracking, this dissertation applied the CAMSHIFT algorithm combining Kalman filtering algorithm to find the search window and the hand postion. CAMSHIFT algorithm is applied to estimates the center and size of tracking window. Then the Kalman filteris used to predict the hand region centroid. Experiments show the improved realtime performance of this method. Finally, a new kind of color matching method was proposed to improve the realtime performance in matching stage.Through testing the system has achieved high tracking efficiency and and robustness. And with the steady perfection of the experiment, this technique will promote the application of fingertip recognition and make human-computer interaction more natural and convenient to flight trainee.