Research On Situations Drive Brain-Computer Interaction Based On AR-BCI

At present,Brain-Computer Interface system based on Augmented Reality(ARBCI)pays more attention to how to analyze more control informations from Electroencephalogam(EEG),but ignores the situationality of stimulus and the operator's insufficient consideration of cognitive characteristics.Aiming at the above problems,this paper introduces the theory of situational awareness,and proposes a situation driven AR-BCI fusion interaction system.Through the design of interactive interface module with machine context awareness and interactive means interface driven by human brain context awareness,the advantages of machine autonomous intelligence and human brain cognitive decision-making are fully exploited.The ARBCI system uses Steady-State Visual Evoked Potential(SSVEP)as the brain computer interface control method to realize the fusion of cognition,decision-making and control in time and space driven by the situation.It obtains the situation information through the situation sensor,and uses deep learning algorithm to perceive and identify the situation components,AR equipment to fuse and display the situation information and perception information,and brain computer interface technology to control and communicate with the outside world.A direct two-way information interaction path is established between the operator and the machine when facing the common situation.The main research work of this paper is as follows:(1)The situation driven human-computer interaction model was proposed.Aiming at the problem of insufficient situational information perception in human-computer interaction system,the human-computer interaction model based on situational driving was proposed,which includes situational perception,situational cognition and humancomputer interaction.It was proved that the combination of BCI and AR virtual reality fusion can make users understand situational information more quickly and accurately,and reduce the cognitive burden.(2)The situation driven AR-BCI interaction system was designed.BCI was used as an interactive method to perceive the user's EEG activities,and the concept of brain computer fusion system is integrated.The machine was introduced into the system to perceive the situation information,and AR glasses are used as interactive devices to display the situation,perception and interaction information.(3)Using SSVEP as the application type of BCI,the feasibility experiments of the proposed interaction model were carried out in two situations,and the designed experimental paradigm was used to verify and test the model.In the virtual situation,AR glasses were used to render 3D models to simulate situational objects.In the two groups of comparative experiments with and without machine situational awareness,the introduction of situational awareness can improve the user's work efficiency and reduce the user's cognitive load through the SSVEP accuracy and cognitive load score.In the real situation,the machine is used to perceive and recognize real objects to achieve more objective.The results show that the situation driven AR-BCI fusion system is feasible and practical in practical application.