| Brain-Computer Interface(BCI)is a method to interact with the outside world by detecting neural signals without relying on peripheral nerve and muscle tissue.As a new human-computer interaction method,the current BCI system is difficult to meet the needs of daily use in terms of command classification accuracy,information transmission rate,and user experience.In this paper,Eye Tracking(ET)technology is combined with BCI technology,and it is hoped that through the fusion of the two modes of physiological signal data,to improve the instruction classification accuracy,information transmission rate and human-computer interaction experience of BCI system.Firstly,in order to achieve the research objectives of this subject,a hybrid BCI synchronous acquisition system based on EEG-ET was built in this paper.The system combined with the EEG amplifier and eye tracker can realize the synchronous acquisition of EEG data and eye tracking data.In the hardware part,we mainly developed the precise event synchronous trigger module which could send event labels to the EEG amplifier and eye tracker simultaneously.In the software part,TCP protocol was used to acquire the data collected by the two devices at the same time,and then synchronized the brain-eye data streams according to event labels and time stamps.Secondly,in order to improve the accuracy of instruction classification of the BCI system and verify the reliability of the established system,this paper carried out a study on the hybrid parallel BCI system based on steady state Visualized Evoked Potential(ss VEP)and ET.Offline experiments and online experiments were designed,and 10 subjects participated in an offline experiment with 40 instruction sets.The experimental results showed that the fusion of ss VEP and ET could improve the classification accuracy and information transmission rate: under 1s data length,the offline average classification accuracy rate and information transmission rate are respectively 99.02%and 156.04 bit/min,which are higher than 85.70% and 122.82 bit/min of ss VEP alone,94.45% and 143.49 bit/min of ET alone.The results showed that the fusion of ss VEP and ET could achieve higher system performance than either of them alone.In order to verify the online function of the system,the data length of 0.5s was adopted in the online experiment,and the maximum information transmission rate of 319.32 bit/min was achieved.Finally,in order to make the human-computer interaction of BCI system more natural,this paper designed a 40-instruction set asynchronous BCI system based on Motor Imagery(MI)and ET,which could realize the autonomous recognition of the task state and the idle state.Four subjects participated in the designated character spelling task and the free spelling task.The accuracy of the controlled character spelling in the designated spelling task was 97.50%,and the correct recognition rate of idle state was 86.30%.The highest spelling rate was 8.21 s/char in the free spelling task and the average spelling rate of the system was 17.91 s/char.The experimental results showed that the hybrid BCI system based on MI and ET could achieve more natural asynchronous output in human-computer interaction,and had advantages in asynchronous BCI with large instruction setsThe research results of this paper show that the combination of EEG and ET can improve the classification accuracy,information transmission rate and humancomputer interaction performance of the single-modal signal BCI.,which provide a new idea for improving the stability and user experience of the BCI system,and realizing the application of the BCI system in daily life. |
