The Research Of Hand Gesture Recognition Based On Harris Corner Detection And Optical Flow

With the advent of digital information age,human beings will leave more and more work for machine to free their hands.To communicate well with machine,more and more researchers engage in this popular field called the human-computer interaction.In PC era,the mainly interaction methods are based on mouse and keyboard.Although it's very accurate,but the limitations of its use is increasingly obvious with the development of multimedia technology and the increased demand of people.Gestures are one of the most commonly used,natural,basic,high feasibility ways in all traditional interaction methods.We can identify a gesture according to its shape and position,but this way can't identify complex gestures,so it cannot be widely used in human-computer interaction field.This paper proposed an algorithm based on the Harris corner detection and optical flow.This method can locate fingertip and track its trajectory,and finial recognize the gesture.the first step of hand gesture recognition is to determine the position of fingertips.The accuracy of fingertips detection directly determines the system accuracy,so it's very important.In this paper,fingertips detection is mainly divided into 4 parts: hand region exaction,Harris corner detection,removing false features,fingertips positioning.Using two-dimensional Gaussian probability model to separate the hand region from background,because skin color has good clustering in YCbCr color model.The gray of fingertips change significantly,so we can extract feature points from binary images by Harris corner detection.And then we use some constraint condition to remove the false feature points.Finally,select the real fingertips with the help of barycenter and convex hull.The second step is to track the fingertip.According to the position of the fingertip and brightness constant hypothesis,calculated the position of fingertip in next frame.Lucas-kanede optical flow performances well in short displacement,which doesn't conform to the actual situation.So we need to introduce the pyramids and iterative algorithm to improve robustness.The last step is gesture recognition.Extract characteristic vector of fingertip trajectory,such as length,width,area,and position.Then identify the gesture by template matching.this paper introduces the details of fingertip detection,track and template matching and proposes a novel and robust gesture recognition method based on Harris corner detection and optical flow.No matter where the fingertip toward is,this method performs well and the experiment result shows that the accuracy is about 85.6%,which lay a certain foundation for human-computer interaction.