Research On Chinese Sign Language Recognition Method Based On Kinect

Sign language recognition is a process of using intelligent devices to translate sign language actions.It is an effective means for hearing-impaired people to communicate with the outside world.But different countries have different sign language expressions.This thesis takes Chinese sign language expression as the research object,and takes into account the advantages of Kinect,which is insensitive to light and background.Based on Kinect,we collect 18 kinds of Chinese sign language of 10 people under different light and angles,and construct a data set composed of 7200 binary images and depth images of hand region,And on this basis,the sign language recognition method is studied.Firstly,in the sign language recognition method based on pattern recognition,the binary image in the data set is preprocessed by median filtering,image morphology and image edge contour extraction.Then,the optimized hand region binary image is extracted by Hu moment feature extraction method,and the extracted features are used as the input of the classifier;Finally,sign language recognition is completed by two classifiers: support vector machine and random forest.Secondly,in the sign language recognition method based on convolutional neural network,this thesis uses the alexnet multi-layer convolutional neural network structure,takes the deep sign language image of sign language as the input,carries on the feature extraction and feature information fusion of deep sign language image through five convolution layers and three pooling layers,and obtains the convolution model with very good training effect.Finally,the full connection layer and softmax are used to divide the sign language image The classifier is used to classify the sign language depth image.Finally,the recognition accuracy and real-time performance of the proposed method are analyzed in the sign language data set collected in this thesis.The test results show that the average accuracy of sign language recognition method based on SVM and random forest is93.4% and 91.7% respectively,and the average accuracy of sign language recognition method based on convolution neural network is 98.3%.