Research On EEG Classification And Real-time Control Based On Motor Imagery

Brain-Computer Interface(BCI)is a new type of human-computer interaction technology that is directly established between the brain and external devices without relying on peripheral nerve and muscle tissue.It can realize direct communication between the brain and the outside world.BCI has gradually become the focus of attention.The BCI system has a variety of classifications according to the mechanism of electroencephalogram(EEG)Among them,the BCI technology based on Motor Imagery(MI)signals,as an active human-computer interaction mode,is more in line with the brain's thinking activities,and is considered to be one of the most promising directions in the BCI system.However,the MI signal has the characteristics of low signal-to-noise ratio,nonlinear non-stationarity and large individual differences,which makes it difficult to analyze the MI signal.Therefore,there are still many difficulties in improving the classification accuracy of MI signals and the practicability of the BCI system.Aiming at the precise analysis and practicability of the MI-BCI system,this paper proposes a multiscale space-time-frequency feature-guided multitask learning convolutional neural network(MSMT-CNN),and designs a real-time control of NAO robots based on motor imaging brain electrical signals.system.The main contents of this paper are as follows:(1)Low classification accuracy for MI signals: Starting from the spatial,time,and frequency characteristics of EEG signals,analyze the correlation between different feature domains and design MSMT-CNN.The model includes four modules: Space-time feature-based representation module(ST)that analyzes the correlation between time and space features;Timefrequency feature-based representation module(TF)that analyzes the correlation between time and frequency domain features;Multimodal fused feature-guided generation module(MFFG)and classification module that merge and re-encode the three features of space,time and frequency.The model is based on multi-task learning,which connects four modules through three tasks and trains and optimizes simultaneously.Using the 2a and 2b datasets and the high gamma dataset of the 4th BCI competition to evaluate the model,the final classification accuracy rates were 81.56%,86.38%,and 95.45%,respectively.By adjusting the combination of different tasks,some modules are rendered ineffective.The confusion matrix and T-SNE are used to quantitatively and qualitatively analyze the effects of different modules.The effects of model fusion and feature fusion on the experimental results are compared.Untrained data is used as the test set to test the degree to which the model eliminates the individual differences of MI signals and causes obstacles in the classification process.Experiments show that the results of this article are still better than other current models in cross-subject experiments.Using different training methods to verify the generalization ability and robustness of the model.(2)Aiming at the practicability of MI-BCI system: This paper designs a set of real-time control system for NAO robot based on the operation of imagined brain electrical signals.An experimental paradigm containing two types of tasks was designed,and a set of MI signal acquisition system was built.For the same subject,using the pre-trained CNN model,guided by the arrow pointing in the experimental paradigm,by wearing an EEG acquisition device,you can achieve online decoding of MI signals and real-time control of the NAO robot.When the subject imagined the movement of the left hand,the left hand of the NAO robot was raised,and when the subject imagined the movement of the right hand,the right hand of the NAO robot was raised.The control system successfully applies the MSMT-CNN proposed in this paper to the real-time control system,which proves the effectiveness of the model proposed in this paper.