3D Fingertip Detection Algorithm And Application Based On Computer Vision

Hand gesture is a more natural and convenient interactive mode comparing with traditional mouse and keyboard devices. Real-time Human-Computer Interaction based on hand gestures plays an important role in both theory and application of virtual reality. The proposed method in this paper can be widely used to implement 3D interaction of video game and 3D mouse.In this paper, we present some key dominant algorithms and implementation of 3D fingertip detection system. Finally, we use the system to implement 3D mouse and some 3D interactive games.The system consists of five parts. The first part is foreground segmentation. We implement an online skin detection algorithm and an improved BGS which combines threshold decision with GMM. Anytime, we can switch between them. Part 2 includes detection of single fingertip and multi-fingertips. We use some algorithms which consist of endpoint search of skeleton, decision of k-vector extreme point and contour characteristic of hand region. Part 3 consists of camera calibration and 3D reconstruction. We use a marker to get world coordinate, then reconstruct 3D position of fingertips. Kalman filter is employed in 3D space to smooth the fingertip trajectory and predict the range of fingertips in part 4. Part 5 implements point, line, face interaction, and we describe some 3D applications using these interaction which include 3D mouse, 3D games. Experimental result shows the efficiency of our algorithm.