The Research On Dynamic Hand Gesture Recognition Based On Video Siamese

Hand gesture is not only a kind of human body language,but also a medium of information transmission.Hand gesture recognition technology refers to recognize the information conveyed by hand gesture,including arm and palm,playing an important role in human machine system,hand sign language recognition,virtual game and intelligent equipment manipulation.It's vital to study hand gesture recognition so as to improve people's life quality and promote the development of image recognition.Dynamic hand gesture refers to the gesture sequence of the hand's appearance or spatial location changing over time.In order to solve the shortcoming of traditional video processing of sampling method and key frame selection method,we present a dynamic gesture recognition system based on video similarity.The main contributions are as follows:(1)A Siamese Neural Network based model used in video sequence siamese analyse.Uniform sampling is in common use in traditional video processing procedure,which can handle some situation.It's likely to introduce noise or burden the classify model when adopts an inappropriate down-sampling value.The Siamese Neural Network based model calculates the similarity score between frames in video,thus reduce the influence of background and redundant frames and generate a higher proportion of useful frame in input downsampling stream.On the basis of siamese neural network which requires a large number of positive/negative training samples,we propose a novel unsupervised positive/negative sample selection method and take gesture type info and appearance info both into consideration.(2)Key frame extraction method based on siamese model,including online sequence key frame extraction method based on sequence decomposition and offline spatial and temporal clustering key frame extraction method based on kmeans.Dynamic hand gesture varies in appearance or position with time goes by.In order to model the variation precisely,we present key frame extraction method based on Siamese model.On the basis of siamese model,the sequence decomposition means to calculate the similarity between the current frame and the preceding cluster.The spatial and temporal clustering algorithm based on kmeans refers to the similarity search in the adjacent time window,which divides all frames of a video into K clusters,with small difference within clusters and large difference between clusters.The Study on Dynamic Hand Gesture Recognition Based on Video Siamese,specially,in order to handle the defect of downsampling method used in traditional video preprocessing procedure,we adopt a distance measurement model based on Siamese Neural Network to calculate the similarity between video frames,and present a key frame extraction algorithm based on the distance measurement model.Experiments show that our method is robust to subject variation and models the dynamic hand gesture well.Good results are archieved on NVIDIA(72.4%)and SKIG(98.1%)datasets.