Research On Gesture Recognition Based On Multimodal Neural Network

Since the beginning of the 21 st century,the rapid development of computers and the widespread application of big data have made human-computer interaction an important part of people's work and life.Among them,gesture recognition is a relatively simple and natural way of human-computer interaction,and it is also a research hotspot in the field of computer vision.Traditional gesture recognition algorithm through good design model and the typical characteristics of gestures to realize,however,the design of the artificial model and gesture feature selection requires researchers have abundant professional knowledge and practical experience to be realized,which greatly improve the threshold,and the artificial design model and gesture feature selection does not necessarily apply to number and a wide variety of dataThesis topic analyzes mainly studies the use of gesture recognition algorithm in neural network environment,analyzes the gesture recognition of domestic and foreign research present situation,the gesture segmentation and feature extraction method has carried on the detailed elaboration,neural network is studied in the application of gesture recognition,using convolution neural network and multiple neural network modal,the characteristics of the multimodal neural network algorithm were described in detail.First put forward the uniform random sampling method based on optical flow,according to the intensity of hand movements to achieve dynamic sampling,and then put forward the hierarchical strategy,internal more local details capture sequences,then use multiple modal data fusion strategy,characteristics of fusion of a variety of data types,thus avoiding single type descriptions of gestures,the use of prior training a good model to extract the convolutional neural network layer and participate in the data as the characteristics of gesture recognition arithmetic.The author carried out a simulation experiment on the proposed gesture recognition algorithm based on multi-modal neural network,built a test environment and carried out a use case test.The experimental results showed that the algorithm could better identify various gestures and had better adaptability and accuracy in each scene.