Research On Myoelectric Multi-dimensional Force Decoding Performance And Simultaneous Control Based On Deep Learning

As a commonly used technology of detecting human motion information,EMG signal has been widely used in the field of smart prosthetic control.However,the current EMG control method still lacks dexterity and has difficulty adapting to a variety of external confounding factors.In addition,users need to be trained for a long time to familiar with the operation of the equipment.Meanwhile,the controller needs to be re-trained frequently for keeping high accuracy during use.These phenomenons hinder the further clinical promotion of smart prosthetic devices.In recent years,multi-DOF simultaneous myoelectric control method has received more and more attention.In order to further improve the practicability of the myoelectric control method,a new human body multi-dimensional force decoding model based on the deep learning is designed in this paper.The model can directly process the raw EMG signals and predict the 3-DOF wrist force states.Through training by largescale dataset,the model has universal adaptability among different subjects.It improves the dexterity of myoelectric control as well as the user experience.Before training this decoding model,it is necessary to label the multi-DOF motions of EMG signals(supervised method).Because the intensity of myoelectric signal has a strong correlation with the magnitude of joint force,the multi-DOF wrist force states are utilized to label multi-channel EMG signals.Currently,only a few studies use joint force information,and these studies need to completely constrain the joint angle,which is quite different from the daily use scenario.In order to obtain reliable supervised samples,a semi-constrained 3-DOF wrist force acquisition system is designed.The 3-DOF wrist force is mapped to the motion(movement)of the cross cursor in real time.In the experiment,the subject’s wrist is free to move,and realtime feedback of the sample label can be obtained by the motion of the cross cursor.Through developing a corresponding data collection paradigm,the acquisition system can establish a unified reference standard among multiple subjects.In this paper,the samples of 11 subjects are collected to form a large-scale data set containing 1.16×106 samples,which reliable data for subsequent model design and experimental analysis.The EMG is sampled at high frequency rate and is susceptible to other confounding factors.The feature extraction method is needed to reduce the dimension of data in the traditional myoelectric control model.In order to avoid the information loss caused by the feature extraction,a deep convolution network structure according to the characteristics of the EMG signal is designed,and the multi-dimensional force information decoding model is obtained in this paper.This model could directly decode the raw EMG signal and predict the 3-DOF wrist force information.In order to promote the model performance,the input data format,model parameters,model prediction targets and other factors are carefully compared and tested.It is found that the knowledge required to predict wrist strength information of target time could be learned from the EMG signals segmented by the longer window.The redundant EMG signal data would not affect the performance of the model.In order to improve the clinical application of the myoelectric control method,it is necessary to test the adaptability of the designed EMG multi-dimensional force information decoding model to various confounding factors.Through the analysis of these confounding factors,they are divided into four levels and the corresponding experimental paradigm is designed.The test results show that the multi-dimensional force information decoding model has higher prediction accuracy under all adaptive levels than the traditional model(SVR).At the same time,the adaptability of the model can be further improved when using large-scale datasets constructed by multiple subjects to train the model.Through the combination with big data,the information extraction ability of deep learning can be fully utilized.The parameters of the underlying convolutional layer of the model have periodicity and channel selectivity,which can further decompose and reorganize the EMG signal,and learn the deep relationship between original EMG signal and wrist forces.In order to obtain further samples,a variety of data-augmentation approaches are designed according to the abnormal scenerio in the EMG signal acquisition experiment.The test results find that three data-augmentation approaches(all the electrode reversal,electrode number placement error and electrode placement misalignment)can further improve the adaptability of the model when they are utilized simultaneously or separately.Although the EMG multi-dimensional force information decoding model could be a general model with off-line accuracy 0.639(R2).But a higher accuracy could be obtained when the model customized for a specific subject.The model fine tuning scheme is designed in this paper.The general model is adjusted by a small number of specific subject experimental data,and a customized EMG control model for a specific subject could be obtained.Through the Fitts’ law online control experiment lasting 28 days,it is verified that the fine-tuned model still has the adaptability of long-term use.Meanwhile,the subject can learn specific muscle synergy through training,and the control effect will be gradually improved during long-term use.Finally,an online experimental platform is built to realize the 3-DOF simultaneous control of the KUKA robot arm and the HIT-V artificial hand based on the model.The completion of a variety of complex operational tasks verifies the effectiveness of the proposed 3-DOF myoelectric simultaneous control method.