Implementation Techniques Of Multi-modal Brain-Computer Systems

After years of efforts on the brain-computer interface(BCI) technology, it is now practical to achieve simple actionless human-computer interaction. However, the communication rates of BCI system which are the bottle neck of the development of human-machine integrative systems based on BCI technology, are very low to be satisfying yet. The reason may be, modern signal processing algorithms are still not strong enough while dealing with the electroencephalogram (EEG) signals which are so complicated and volatile. There is a common sense nowadays that single brain monitoring technique(EEG) is not enough to realize efficient brain-machine integrative system, and the ideas of multi-modal measuring which combine several technologies will be the trends.Based on the above ideas, we applied for the support of basic research program of NUDT "human-machine integrative system based on multi-modal measurement" and have been approved. In the context of this project, this paper investigated several non-injured brain activity measurements techniques, especially the electroencephalogram(EEG), electromyogram(EMG) and functional magnetic resonance imaging(fMRI) which have been the three key issues for realizing our own human-machine integrative systems, and analyzed the different characters of these signal in the sense of temporal-frequency feature, resolution and signal-noise ratio. After reviewing the development of typical EEG-based BCI systems in the literature, we proposed a new EEG based BCI experimental paradigm of feedback controlling to exploit human's outstanding ability of balancing and self-adaptivity, which is superior to those typical BCI paradigms where human act only as the signal source. We applied virtual inverted pendulum control system as a typical experimental platform of the new BCI paradigm, with which we can not only experiment on BCI, but also perform new experiment with EMG feedback. This platform is appropriate for multi-modal EEG/EMG fusing, on which we designed three groups of EMG data acquiring experiments in addition to the experiment of EEG signal processing. Analysis on the acquired data revealed that the high SNR EMG signal is easy to obtain and suitable for real-time processing, which is appropriate for efficient human-machine integration. Moreover, this paper also investigated the basic technique of EEG/fMRI fusing, and point out it may be a good way for solving the inverse problem of locating EEG sources with the high spatial resolution of the fMRI images, which provides ideas for studying the brain mechanism of harmonious motion controlling and may act as fundamental theories for the realization of human-machine integration.