Research On A Human-Machine Interaction Device By Information Fusion Of EEG And Blood Oxygenation On Forehead

Human-Computer Interaction (HCI) technique can provide direct communication and control channels between human and computer. When the connected position is human brain, the user can control computers or other electronic devices only on user's mind, which becomes a Brain-Computer Interface (BCI). Currently, the trend in the development of BCI research is not limited to using only one kind of physiological signal, such as BCI based on EEG. BCI systems based on other kinds of signals which can reflect the state of brain thinking activities, such as Magneto encephalography (MEG), functional Magnetic Resonance Imaging (fMRI), functional Near Infrared Imaging (fNIRI) and so on are now being researched. Besides, the research of BCI technology based on information fusion of different kinds of information of brain thinking activity comes forth gradually.In this thesis, the method of monitoring the blood oxygenation changes of brain tissue which reflects the status of brain functional activity, by the means of Near Infrared Spectroscopy (NIRS), was attempted to combine with the existing EEG- based BCI system research, in order to gain a better method of HCI reflecting brain thinking activity by the fusion of two kinds of information. For this reason, this thesis designed a HCI device by information fusion of EEG and Blood Oxygenation on forehead. Through the subjects taking a thinking task experiment of specified pattern of left/right hand imagery tasks which evoked the changes of EEG and Blood Oxy- genation signals, this device could achieve real-time synchronization acquisition of EEG and Blood Oxygenation signals (corresponding to the functional area of Prefrontal Lobule), then the methods of feature extraction and pattern recognition were induced to achieve the fusion of EEG and Blood Oxygenation information, which prepared for the control of exterior electronic devices of HCI device.In this paper, time-frequency analysis based on Short Time Fourier Transformation was induced to analyze the time-frequency characteristics of the ERD/ERS of dynamic EEG signal evoked by left/ right hand imagery movement. Then Fisher Separability Analysis was used to identify the special characters of power spectral density distribution, which turned out there was no significant difference between left and right hand imagery tasks while the difference between the imagery and resting periods was very significant. Simultaneously, the methods of frequency domain and time domain analysis were separately used to analyze the frequency band and changes in time domain of Blood Oxygenation signal. After the significant difference analysis based on the statistical Student's T Test, turned out a similar conclusion with EEG. Therefore, the features final extracted were the differential features between the imagery and resting periods of the target tasks. In the recognition of imagery and no-imagery tasks, the method of 5-fold cross validation based on Support Vector Machine was induced to perform feature recognition based on two kinds of single-feature of EEG and Blood Oxygenation, and fusion-feature gained by feature level fusion of them, which gained the results of recognition.Results of this study showed that the mean recognition rate of fusion-feature could reach from 75.04% up to 91.11%, which was better than that of single-feature of EEG and Blood Oxygenation, to some extent. This demonstrates that the method of HCI based on the fusion of EEG and Blood Oxygenation information is hopeful to improve the recognition rate of this system, which is worthy of more in-depth study.