Research On The Key Techniques For The Hybrid Brain Computer Interfaces Based On MVEP And MI

The hybrid brain-computer interface(BCI)is one of the research hotspots in the current BCI field.It can compensate for some deficiencies of the single-modality BCI through the combination of different single-mode BCIs,which can improve the performance of the BCI system in terms of flexibility or efficiency.In this dissertation,we constructed a new hybrid BCI system by combining two different single-modality BCI signals,which are motor imagery(MI)and motion-onset visual evoked potential(mVEP),respectively.And the former belongs to the spontaneous signal of subjects and the latter is evoked by outside stimulus.The spontaneous BCI does not require the external stimuli,but only requires the subjects to imagine to control.Conversely,the evoked BCI needs external stimuli exerted to subject instead of the subjects' imagination.Based on different mechanism of the two BCI signals,we combined the two kinds of information by requiring the subjects to perform the corresponding two different tasks simultaneously.Thus we could realize the BCI system which could perform two single-modality tasks simultaneously to obtain more flexible and efficient control.Meanwhile,because the stimuli way of mVEP is relatively much softer,which has no sudden change of the luminance or a high contrast of visual objects,the subjects may experience less visual fatigue and feel more comfortable when using this hybrid BCI system.However,the hybrid BCI also faces the same problems as the single-modality BCI,such as the background noise interference and the stability of EEG signals,which influence the BCI performance in a certain degree.To further improve the performance of the hybrid BCI system,we optimized the performance of the two single modality BCI systems that are used to construct the hybrid BCI.The main content of this disseration is as follows:1.Develop a new hybrid BCI system by combining MI and mVEP to realize two-dimension(2D)motion control of the cursor.The results of multi-modality tasks show that the proposed hybrid BCI system could evoke the expected MI and mVEP features simultaneously,both of which are very close to those caused by the corresponding single-modality BCI task.Moreover,the results also show that the proposed hybrid BCI system can provide more effective and natural control commands for the 2D motion control.These results consistently confirmed the feasibility of our proposed hybrid BCI system and could realize more efficient 2D motion control than the single-modality BCI system.2.As for how to efficiently extract the mVEP features in the hybrid system,we propose a new feature extraction method by combing the deep learning(DL)and compressed sensing(CS)methods to improve the mVEP-basded BCI performance.In the experiment,we use the data collected from subjects to compare the results obtained from the new and traditional amplitude-based mVEP feature extraction methods.And the results show that the new method based on the combination of DL and CS can extract the mVEP features more effectively,resulting in an obvious improvement of accuracy for mVEP based BCI system.3.In order to realize the effective online-update learning of mVEP information in the hybrid system,we propose an adaptive calibration framework for mVEP-basedded BCI system.The core of the framework is the combination of support vector machine(SVM)and fuzzy C-means clustering(fCM)to find the reliable information in new samples to adaptively update the training set,so that the state changes of the subjects can be effectively tracked.We also use the data collected from subjects in experiments to verify the adaptive calibration framework,and the results show that compared with both the traditional CSP method and the single mode adaption method,the proposed adaptive calibration framework based on SVM and fCM has a better classification accuracies,which reflects the effectiveness and efficiency of the proposed adaptive calibration framework.4.As for how to efficiently extract the MI feature in the hybrid system with less training efforts,we adopte the semi-supervised idea to improve the CDBN feature extraction method,and propose a new feature extraction method called SSL-CDBN for left-and right-hand MI tasks classification problems.In this method,we add the label information of samples into the unsupervised training process and reduce the excessive reliance on the labeled information by adopting the clustering information of the labeled sample instead of directly using the labeled sample itself,thus avoiding the limitation of network generalization capability.In the experiment,we use the data collected from subjects to evaluate the potentials of the proposed SSL-CDBN method for cross-subject classification problems.And the results show that without training,the average accuracy of the SSL-CDBN method improved 4.1% and 3.0%,compared with the CSP and CDBN methods,respectively.The grouped experiment results also show that the SSL-CDBN method has better improvement effects on the poor-performance BCI group,which has very important potential value in the practical BCI application.These results show that the SSL-CDBN method could effectively extract the features of two different types of structures of the MI data collected from the new subjects based on the information of the old existing subjects,and thus effectively improve the performance of MI-BCI and significantly reduce the subjects' training burden in the practical application of MI-based BCI.In summary,we propose and build a hybrid BCI system based on MI and mVEP.And then we carried out a series of studies to improve the performance of the single-modality BCIs that make up the hybrid BCI system by improving and proposing the feature extraction method and recognition method which are more stable and effective for these single-modality BCIs.Those works in dissertation may provide the potential technical basis for the further development of high performance hybrid BCI system.