Research On The Saccadic Eye Movement Recognition Based On The Combination Of Electrooculography And Videooculography And Its Application

Human-computer interaction(HCI)refers to the process of information exchange between person and computer.Because the eye movement signal has the advantages of strong controllability and easy to collect,the design and development of the human-machine interaction system based on eye movement has received more and more attention.Some produces such as mouse control system,wheelchair control system and virtual keyboard system.have entered into people's live.Generally speaking,according to different functions,eye movement signals can be roughly divided into four categories:saccade,gaze,smooth follow,and blink.Among them,the saccade signal as the most frequent eye movement behavior,the diversity of the saccade direction will be beneficial to generate more control commands.Therefore,the use of saccade signals for human-computer interaction has become a new research hotspot.Nowadays,from the perspective of the acquisition of saccade signals,Electrooculography(EOG)and Videooculography(VOG)are the two main methods of recording eye movement information.Among them,the EOG method has the advantages of low cost,low computational complexity and insensitivity to light.But this method will cause serious noise interference due to the micro-shift of the electrode;the VOG method can overcome the problems of the EOG method to a large extent.And it has the advantages of high eye tracking accuracy and easy portable design,but the method is susceptible to interference from external light.Considering the advantages and disadvantages of EOG and VOG signals,in order to improve the correct rate of saccade signal recognition,this thesis proposes a method combining EOG and VOG to identify the saccade signal.On this basis,a Chinese eye writing system based on dual-modality is designed and implemented.The specific works of the thesis include:(1)We investigate the current state and development trend of human-computer interaction system based on saccade signal.On the basis of this,we also survey the saccadic signals recognition method associated with the processing,endpoint detection,feature extraction,and classification.At the same time,the advantages and limitations of EOG and VOG in the recording of eye movement information were compared and analyzed.(2)We propose a recognition method combining EOG and VOG.In the stage of saccade signal acquisition and preprocessing,eye movement data of EOG and VOG according to eight directions are acquired simultaneously.Furthermore,we use the sliding window combined with the short-time energy threshold to detect the starting point and the ending point of the saccade signal.The wavelet packet coefficients and the two-dimensional wavelet features are extracted as the feature vectors of the saccade.In the saccade identification phase,two modal fusion methods are used to identify the saccade signal.One is that based on the feature level fusion method,it combines two feature vectors of EOG and VOG,and uses Support Vector Machine(SVM)to classify the direction of the saccade;the other is based on the fusion method of decision level,which combine the classification results obtained by EOG and VOG data to get the saccade recognition results.The experimental shows that these dual-modality methods have higher recognition accuracy than single modality.(3)We propose a simplified Chinese eye-writing system based on dual-modality.Firstly,according to the stroke characteristics of Chinese characters,we classify all of Chinese stroke types as five categories:horizontal,vertical,left-falling,right-falling,and turning,and design the corresponding eye movement patterns.Secondly,we synchronously collect the EOG and VOG data of the stroke template data and the Chinese stroke data,then detect the stroke segment with the sliding window combined with threshold method.Considering the difference of data length and the complexity of the algorithm,this thesis uses the DTW method to classify the eye-writing stroke segments of Chinese characters,and uses the maximum fusion method to classify the EOG and VOG signal data.In order to correctly predict Chinese characters,we calculated the cross entropy of the input stroke order and the coding order of Chinese strokes in the dataset,and used this as the basis for recognizing Chinese characters.Based on the above method,a Chinese-based eye writing system is designed.Experimental results show that the system can effectively input eye writing characters in a short time and has high prediction accuracy.