A Study On Multi-modal And Multiple Degree-of-Freedom Brain-computer Interfaces

Brain–computer interfaces provide non-muscular communication and control bydirectly translating brain activities recorded from the scalp into computer control sig-nals and thus enable users with motor disabilities to convey their intent to the externalworld. An important issue in BCI research is multidimensional control, which enablesconsiderably enhanced interfacing between the user and the machine. Potential applica-tions include BCI controlled computer mouse, wheelchair, neuroprothesis, etc. To date,most of the BCIs for multidimensional control have been invasive or with expensive neu-roimaging devices like magnetoencephalography. On the other hand, the developmentof noninvasive EEG-based multidimensional control BCI is impeded by the difficulty inobtaining multiple independent control signals from the noisy EEG data of poor spatialspecificity.Several studies have shown that hybrid BCIs may yield a better performance thanBCIs that use only one type of brain signal. The main goal of this study is to developmulti-modal (hybrid) BCIs and to provide multiple degree-of-freedom (DoF) output, thusprovide multidimensional control, by them. Emulating a computer mouse with BCI is atypical example. Here, the first task is to move a mouse from an arbitrary initial positionto an arbitrary target position based on two independent signals from EEG, which arefor the vertical and horizontal movement of the cursor respectively. The second task isto select the target-of-interest and reject the target-of-no-interest, as fast and accurate aspossible.By combining P300and mu rhythm, a hybrid BCI system for2-D cursor movementcontrol is first proposed. The BCI system simultaneously performs detection of P300andmotor imagery, and uses the detection outputs to control the cursor’s movement in a2-Dworkspace. The users move the mouse up/down by attending to one of the three flashing"UP"/"DOWN" buttons in a specially designed GUI. Meanwhile, they move the mousetoward right by imagining his right hand and vice versa. Once the cursor hits a target,the system makes a decision of selection/rejection of the target. The target selectionis implemented by combining motor imagery and P300into a hybrid feature, which isfurther classified to make the decision. If the target is of interest, the user can selectit by focusing on the flashing "STOP" button on the left side without motor imagery. Otherwise, he can reject it by motor imagery without focusing on any flashing buttons.In this way, a BCI mouse was implemented.Based on the BCI mouse, we developed a BCI browser, by which the user can selectan intended target in a multi-target web page. A target iflter based on the adaptiveP300speller (for keywords input) was introduced to exclude most of the targets-of-no-interest. A series of mouse movements and selections/rejections were performed to selectthe intended target. Seven healthy subjects attended the online experiment and all ofthe subjects can open a web page and select items-of-interest in it using our system.We also proposed a BCI mail client for electronic mail communication based on thehybrid BCI mouse. The adaptive P300speller was embedded into this system for effectivetext input. Six subjects attended the online experiment and successfully received，read,wrote emails and attached ifles with this system.An adaptive P300speller is then proposed. The P300speller adaptively selects thenumber of epochs to average, according to the subject’s current performance. With thisP300speller, we won the championship of the Character Input event in the1st ChinaBCI Competition (2010). Our subject entered72characters in6minutes with1error.