Multi-sensor Gesture Design And Recognition Method Based On Wearable Devices

With the development of computer software and hardware technology,the computing speed of computer has been rapidly improved.Nowadays,the way of human-computer interaction has become the factor of limiting the development of computers.In recent years,many new interactions have been proposed such as voice,electrooculogram,electroencephalogram and gesture.Gesture has been the main way of interaction between humans and tools since ancient times,which provides an intuitive,natural and effective interaction.Therefore,the research of gesture interaction has great significance for human-computer interaction.At present,the research on gesture interaction mostly focuses on the recognition accuracy,while the user-centered methods of designing gesture are ignored.It limits the applications of gesture recognition algorithms in practical products.In the research of gesture recognition algorithm,there are few researches on the gesture set of mixed static gestures and dynamic gestures.And the algorithms usually use a single signal,which limits the accuracy of recognition.Therefore,in the scenario of using gesture control drone operation,this thesis designs a user-centered gesture set.A wearable gesture recognition device is constructed by myoelectric sensors,flex sensors,an inertial sensor and an embedded circuit board.The gesture recognition algorithm is based on the EMG signal,the deformation signal and the inertial signal,which is implemented on the wearable device.Finally,the algorithm is verified by a drone control simulation software.The main results of this thesis are as follows: 1.This thesis builds a wearable gesture recognition device upon multi-sensor.The device uses myoelectric sensors to collect EMG signals of forearm,uses flex sensors placed at fingers to collect the deformation signals generated by the finger bending,and uses the inertial sensor placed at the wrist to collect the motion signals of palm.A glove-type wearable device is composed of sensors,collecting circuits and an embedded signal processing module.The wearable device completes the function of collecting and recognizing of gesture signals and interacting with others.2.This thesis designs a user-centered gesture set in the background of using gesture control drone operation.Firstly,the drone functions in the market are investigated and 15 common functions are selected.The experimenters are invited to design gestures based on 15 drone functions.Then other experimenters complete a questionnaire.According to the experimental results,15 gestures are customized according to the user identification.The user-designed gesture sets are evaluated from the fit of each gesture to the drone function and the comfort of the gesture,and the usability and user experience of the gesture set are analyzed.The application of gesture recognition algorithm in practical products is promoted by this work.3.A multi-sensor gesture recognition algorithm based on EMG signals,deformation signals and inertial signals is proposed.The algorithm completes the processing and classification of gesture signals.Firstly,the algorithm detects the active segment of gesture signals and determines its validity.Then an effective and complete gesture signal is obtained.Secondly,the wavelet transform and the Gaussian filter are selected to reduce the noise of gesture signals,and then the quaternion is used for the coordinate transformation of inertia signal.For 15 gestures,the accuracy of the fusion weighted KNN and GMMHMM-SVM algorithm is 97.33%,and the accuracy of CNN algorithm is 92.13%.In this thesis,comparing two algorithms,the algorithm with high accuracy is selected to be transplanted into the wearable gesture recognition device.Finally,the interaction between the device and the drone control simulation software is completed.This thesis designs a wearable gesture recognition device,which includes a data glove,a data armband and an embedded signal processing module.The wearable gesture recognition device and the customized gesture set provide a more natural and humanized way for human-computer interaction.It overcomes the influence of the use environment and can be applied to the field of robot control,smart home,interactive entertainment,VR/AR and other potentials interactive application.