Research On Human-computer Interaction Methods Based On Eye Movement And Motor Imagery

Human-computer interaction refers to the process of exchanging information between human and computer in a certain way for completing a task.In recent years,the application potential of human-computer interaction technology emerged in those application areas,such as wearable computers,immersion game equipment,VR platform,telerobot and telemedicine system.Since single signal channel is usually applied in traditional human-computer interaction system,such as voice or gesture,the interactive mode is simple and has limited application area.In order to improve the performance of human-computer interaction system,the main work of this article is to design a multi-channel interaction system which combines EOG and EEG to realize real-time control of robot action based on the signal acquired with the Emotiv EPOC+ headset.EOG and EEG identification is the basis of realizing multi-channel human-computer interaction.In this paper,an adaptive threshold normalization method is proposed to extract features of EOG and feature matching method is used to recognize 6 eye movements acquired from 6 experimenters,which contains looking upward and downward,looking left and right,blinking eyes twice and blinking eyes three times.After testing,the recognition rates of off-line eye movement signals in vertical and horizontal directions reach 97.3% and 99.99% respectively,and the average recognition rate reaches 98.6%.When someone imagines the movement of his left hand or right hand,the phenomenon of ERD(Event Related De-synchronization)and ERS(Event Related Synchronization)will appear on motor areas of his brain on both sides.Based on this phenomenon,the features of power spectrum,short-term energy and cumulative energy of EEG are extracted,and BP neural network is used to classify the EEG signals.Finally,the maximum off-line recognition rate reaches 92.86%.Benefitting from this simple feature extraction algorithm,the delay in the process of real-time controlling robot can be reduced effectively.To realize the real-time interaction with robots,Emotiv EPOC+ headset was chosen to be worn by experimenters to acquire EOG and EEG signals.After being processed with MATLAB,the recognition results were sent to the NAO robot to control the movements of his blinking eyes and rising hands by a communication program which is written with Python.The experimental results show the multi-channel human-machine interaction method is effective and lay the foundation for future intelligent applications.