Air Handwriting Recognition Technology Based On Depth Image

The traditional human-computer interaction is realized through some input devices such as mouse and keyboard,which limit the speed and naturalness of interaction and can no longer meet the needs of users for more intelligent interaction.Gesture interaction is simple,visual and intuitive,so it is widely used in various fields.However,the existing gesture technology cannot achieve three-dimensional text input,and traditional handwriting recognition input relies on a touchpad.Voice interaction can only meet part of the operation requirements,and cannot be fully qualified for the work of 3D space text input.New interactive methods are urgently needed to realize 3D space text input.Air handwriting trajectory recognition allows users to write in the air in a natural and unconstrained manner,thereby providing a more convenient and comfortable interactive experience.Thus,handwriting in the air has gradually become a research hotspot.The main research contents of this thesis based on air handwriting trajectory recognition technology are as follows.Firstly,this thesis improves the traditional fingertip detection method based on convex envelope,aiming at the problems such as the vulnerability of traditional fingertip detection to the interference of light and complex environment,missing detection and false detection.First,hand images are obtained through RealSense;Then,based on the traditional convex hull detection,the distance between convex hull and adjacent common points of the contour is calculated and the threshold value is set to optimize the points with too close interval.Then the parallel vector threshold was set to further screen out the fingertip through the feature of approximate parallel line on both sides of fingertip contour.At the same time,this thesis introduces a writing gesture detection mechanism.When the finger enters the detection blind zone,the point closest to the camera on the hand is extracted as the fingertip point.Finally,the Kalman filter algorithm is used to estimate the position of the fingertip,and the optimal position of each frame of the finger is determined by combining the observation data to realize the real-time and stable tracking of the finger.Secondly,to solve the problem of large computational load and low recognition rate of traditional DTW algorithms,this thesis proposes a new trajectory recognition method by optimizing the DTW algorithm.First,set the search boundary through the slope to limit the path search range;second,when calculating the distance between the trajectory to be recognized and the template,the distortion threshold is set to eliminate the template with a large difference in distance,which reduces the amount of calculation;finally,KNN and DTW are combined to realize the classification of finger fingertip trajectory and improve the recognition rate of the algorithm.Thirdly,on the basis of the above research,a prototype system of handwriting trajectory recognition in the air is designed and implemented.This system collects data information by RealSense,and realizes the function of handwritten track recognition in the air through fingertip detection technology and track recognition technology studied previously.