A Study On EEG- And EOG- Based Asynchronous Human-computer Interface And Its Application

Human-Computer Interaction(HCI)refers to the interaction between human and computers or other external equipments.Different human-computer interaction technologies are equivalent to different bridges between human and machines.However,the conventional “bridges” such as the keyboard,mouse,voice,touch,etc.usually can not be used by the people with severe disabilities,such as amyotrophic lateral sclerosis(ALS,also known as the “frozen person”),spinal cord injury(SCI),stroke,etc.Extro-ordinary HCI systems based on bioelectrical signals provide a perfect alternative for these people.This thesis proposes several practical asynchronous HCI systems based on electroencephalogram(EEG)and electroretinography(EOG)signals.First,we consider a typically asynchronous system,i.e.,brain switch,where the key issue is to detect the control and idle states in an asynchronous manner.Most existing methods utilize a threshold to distinguish the control and idle states.However,it is often time consuming to select a satisfactory threshold,and the chosen threshold might be inappropriate over a long period of time due to the variability of the EEG signals.This thesis presents a new P300-based threshold-free brain switch.Specifically,one target button and three pseudo buttons,which are intensified in a random order to produce P300 potential,are set in the graphical user interface(GUI).The user can issue a switch command by focusing on the target button.Two support vector machine(SVM)classifiers,namely,SVM1 and SVM2,are used in the detection algorithm.During detection,we first obtain four SVM scores,corresponding to the four flashing buttons,by applying SVM1 to the ongoing EEG.If the SVM score corresponding to the target button is negative or not at the maximum,then an idle state is determined.Moreover,if the target button has a maximum and positive score,then we feed the four SVM scores as features into SVM2 to further discriminate the control and idle states.Several experiments including a real wheelchair control experiment are conducted with eight healthy subjects and five patients with SCIs.The experimental results not only demonstrate the effectiveness of our approach but also illustrate the potential application for patients with SCIs.Recently,not only EEG,EOG signals are also widely used to develop switches.However,most existing EOG-based switches are characterized by disadvantage of high false positive rates(FPRs),because EOG signals are highly affected by unintended/spontaneous eye movements.We develop a novel EOG-based switch design of which the GUI includes a switch button that flashes once per 1.2 s.The user is instructed to blink synchronously with the flashes of the switch button to issue an on/off command while the single-channel EOG signal is collected.A waveform detection algorithm is applied to the ongoing EOG signal,which discriminates the intended and unintended blinks mainly based on the synchrony between the blink and the switch button's flash.As an application,the proposed EOG-based switch is used to produce start/stop commands for a wheelchair.Several online experiments are conducted with ten healthy subjects.An average accuracy of 99.5%,an RT of 1.3 s for issuing a switch command in the control state,and an average FPR of 0.10/min in the idle state are achieved.The experimental results are better than that achieved by the abovemetioned brain switch,and therefore demonstrate the effectiveness of the proposed EOG-based switch.Next,inspired by the proposed EOG switch,we develop a novel single-channel EOGbased speller that allows users to spell asynchronously by only blinking.Forty buttons corresponding to 40 characters displayed to the user via a GUI are intensified in a random order.To select a button,the user must blink his/her eyes in synchrony as the target button is flashed.Two parallel data processing procedures,specifically SVM classification and waveform detection,are combined to detect eye blinks.Decisions are made based on the results of SVM classification and waveform detection.Three online experiments are conducted with eight healthy subjects.An average accuracy of 94.4% and an RT of4.14 s for selecting a character in synchronous mode,as well as an average accuracy of93.43% and an FPR of 0.03/min in the idle state in asynchronous mode are achieved.The experimental results therefore demonstrate the effectiveness of this single-channel EOG-based speller.Lastly,based on the combination of EEG and EOG signals,we propose a new asynchronous hybrid brain-computer interface(BCI)system,which integrates a speller,a web browser,an e-mail client,and an explorer.The speller subsystem is similar as the abovemetioned EOG speller,in which the user can spell a character by blinking eyes in synchronous with the corresponding button's flashes.In the subsystem of browser/e-mail client/explorer,the user can control the horizontal movement of the mouse by imaging left/right hand motion,and control the vertical movement of the mouse,select/reject a target,or input text in an edit box by blinking eyes in synchrony with the flashes of the corresponding buttons on the GUI.With this hybrid BCI system,the user can surf the internet,input text,send/receive e-mails,and mampulate files.The hybrid BCI system is tested on five healthy subjects,and the experimental results demonstrate the effectiveness of the proposed system.