Isolated Word Recognition In Sign Language Based On MYO Armband

Sign language allows people with hearing and speech disabilities to communicate smoothly,but it is still difficult to solve communication problems between people with hearing and speech disabilities and other groups.Therefore,the sign language translation technology that transforms sign language into phonetic words can help people with hearing and speech disabilities to better integrate into society and realize their value more easily.As the core technology of sign language translation,gesture recognition is the main research content of this paper.This paper first introduces the basic principles of gesture recognition,how to extract gesture features and the algorithms commonly used in the field of gesture recognition.The gesture recognition algorithm introduces hidden Markov model,BP neural network and convolutional neural network.Secondly,the collection of surface EMG signals,acceleration signals and gyroscope signals was achieved using the innovative MYO armband from Thalmic Labs,Canada.The MYO armband can transmit these three types of signals through low-power Bluetooth.It is not only easy to wear,but also more mobile during use.It is also more suitable for scenes that require long-term wear such as sign language recognition.The principle of surface electromyography signal generation,acceleration signal sensor and gyro sensor and program flow for collecting data using MYO armband are introduced.Then,for the problem that the two-dimensional convolution kernel and feature map in the two-dimensional convolutional neural network are not suitable for the recognition of surface EMG signals,we choose a more suitable one-dimensional convolutional neural network.Experiments show that the given one-dimensional convolutional neural network has a higher accuracy rate and can effectively identify static sign language isolated words.Finally,the recognition of surface EMG signals can not well characterize the motion characteristics of gestures.The acceleration flow and gyroscope flow are introduced,and the three-branch structure adapting to the data situation,namely multi-stream convolutional neural network,is given.Experiments show that the multi-stream convolutional neural network has higher accuracy of data stream discrimination,and compared with the over-fitting of ordinary multi-dimensional convolutional neural networks for multi-stream data,this is related to The over-fitting of the input data with strong noise is similar.In this paper,the highly integrated MYO armband is used,and the surface EMG signal is used as the main signal for hand recognition.The acceleration signal and the gyroscope signal are used to characterize the motion trajectory of the arm and the motion characteristics such as flipping,and the multi-flow volume adapted to the data is given.The neural network is used for classification and identification.This identification method has two main characteristics: First,the selected convolutional neural network can directly use the original data,and automatically extract features,without relying on artificial design methods to extract features,which not only avoids complexity and is very computationally intensive.The large signal preprocessing part solves the problem of the limitations of feature selection.Secondly,aiming at the characteristics of multiple elements of dynamic sign language isolated words,we give the three-branch multi-stream convolutional neural network structure of muscle current,acceleration flow and gyroscope flow.Compared with the traditional convolutional neural network,it is impossible to remove the difference characteristics between data and thus over-fitting can not get a good recognition effect.The multi-stream convolutional neural network shows when dealing with the signals of multi-class data representation.Good performance,will not take the difference between the data as a feature,can effectively extract the characteristics of each data,can effectively identify dynamic sign language isolated words.