Research On Augmented-Reality Visually Stimulus Paradigm And Its Asynchronous BCI System For Prosthetic Hands

With the excellent human-machine coupling characteristics,the alternative prosthetic hand system based on Brain-Computer Interface(BCI)technology has gradually become an emerging research hotspot.Steady-State Visual Evoked Potential(SSVEP)-based BCI is more suitable for prosthetic hand systems due to the advantages of light system,less training process and high information transmission rate.However,the existing SSVEP-BCI still has certain limitations in controlling focus,system interaction logic and portability.Therefore,to improve the asynchronous application capability and portability of the BCI system,an augmented reality(AR)technology was adopted to conduct the research on the AR visual stimulus paradigm and its asynchronous application system for the prosthetic hand system.Aiming at the control focus of the traditional SSVEP paradigm,AR was introduced into the prosthetic hand system.Based on the generation mechanism of EEG signals under AR visual stimulus,the AR visual stimulus paradigm and stimulator design are carried out.Afterwards,feature analysis of the EEG signals under AR visual stimulus is performed.Experiments show that in a specific scene,compared with the LCD stimulus paradigm,the AR visual stimulus paradigm can induce more obvious SSVEP EEG characteristics,and the spectral amplitude and signal-to-noise ratio at the stimulus frequency are improved by 17.41% and3.52%,respectively.In view of the decoding of EEG signals and the asynchronous interaction of the system,the application scenario of continuous manipulation of the prosthetic hand was focused on and conducts the following two research.1)In order to improve the decoding accuracy and generalization of AR-SSVEP signals,the stimulus pattern recognition of EEG signals is realized based on the Extended-CCA algorithm.Experiments show that the algorithm has better decoding performance under shorter data analysis time,and the AR visual stimulus paradigm has higher EEG decoding performance than the LCD stimulus paradigm.2)In order to improve the asynchronous level of the system,this study proposes the CenterECCA-SVM algorithm based on the attentional orienting logic of the AR visual stimulus paradigm to realize the idle state detection of AR-SSVEP signals.Through the analysis and evaluation of discrete and continuous visual stimulus experiments,the optimal model of the algorithm achieves outstanding asynchronous pattern recognition accuracy.To improve the human-machine collaboration logic of the prosthetic hand system,the stimulator switching method based on object detection and behavior judgment was studied in this paper,which adopts the YOLOv4 algorithm and has gone through the formulation of the stimulator switching strategy and the construction and training of the object detection model.The detection model achieves a mean average precision of 90.8%,an average FPS of 20.29,and an average object detection confidence of 96.4% for the interaction scene of an eightdegree-of-freedom prosthetic hand.The model has excellent CPU usage,recognition speed and detection performance,and is integrated with the AR visual stimulator to achieve smooth prosthetic hand interaction detection and stimulator switching.To verify the overall performance of the above paradigms and methods,the construction and experimental verification of the intelligent asynchronous portable brain control system were carried out.In terms of software system,a multi-process-thread concurrent software architecture with AR visual stimulator thread,object detection and behavior judgment thread,asynchronous EEG signal detection thread,and EEG online continuous acquisition thread as the core is constructed.In terms of hardware system,the component selection and system design were carried out,and an intelligent asynchronous portable brain control system based on AR visual stimulus paradigm was built.Then four kinds of daily operation tasks including pouring-drinking water,opening the door,writing,and typing are selected to carry out the online experimental verification of the system.The results show that: the accuracy rate of idle state detection of AR-SSVEP EEG reaches 95.65%,and the accuracy rate of stimulus pattern recognition reaches 93.48%.The 8-DOF prosthetic hand has an average object detection confidence of 97.1% and an average FPS of 16.4.The user successfully completed the four operation tasks,and the average time consumption met the daily acceptable range.