Research And Implementation Of Vision-based Mobile Tiny Gesture Trajectory Recognition

With the continuous development of human-computer interaction technology and the popularity of mobile devices,non-contact gesture recognition technology for mobile terminals has gradually attracted attention.Gesture recognition technology has been developed for many years,but there are still several difficulties with vision-based tiny gesture trajectory recognition for mobile terminals:(1)Complex background and interference of light have great interference to gesture recognition;(2)Limited computing resources of mobile devices;(3)Fingertip detection methods have poor performance,and there are many problems such as error detection of bending fingers and interference of false points;(4)With habits of performing gesture varying with users,the direction and amplitude of the performed gesture trajectory are inconsistent.The trajectory is different even if the same user repeats the same trajectory at different times.The difference in time and space of gesture trajectory increases the difficulty of hand trajectory recognition.For the problems existing in mobile tiny gesture trajectory recognition,the research focuses on three stages of fingertip detection,feature extraction and trajectory recognition.The main work is as follows:1.In the fingertip detection stage,this paper proposes a real-time fingertip detection method based on the improved centroid distance method.Firstly,a threshold of the centroid distance of the fingertip is used to eliminate interference of bending fingers.Then calculate the weight of the fingertip in the four directions of up,down,left and right to judge the direction of the palm.Finally,the non-fingertips are excluded based on the positional relationship between the candidate fingertips and the centroid.This method has high robustness in complex environments and good performance on calculation speed.2.In the feature extraction stage,this paper proposes a feature extraction method based on the central point direction angle.The directional angle features of each finger trajectory point relative to the central point are extracted to form a tiny gesture trajectory feature sequence.This method can maintain the feature difference whilst decreasing the feature extraction time.3.In the trajectory recognition stage,this paper proposes a trajectory recognition method based on improved LSTM model.First,aiming at the problem of weak aggregation of the Softmax loss function of the traditional LSTM model,the smallest intra-class variance and the largest inter-class variance are achieved by normalizing and setting appropriate cosine coefficients and cosine margins.As a result,the discriminative power of the model is improved.Then,an adaptive model training method is proposed to find the best parameters of the model.Defining the gradient value of the loss and compare the gradient values to determine whether the model converges to reduce the model training time.And find the best parameters of the model according to the model's iteration number and accuracy.Eventually,a model on the hand trajectory feature sequence will be arranged based on the improved LSTM model.