An Evaluation Method Combining BubbleView With ERP For Graphical Interfaces In Human Machine Interaction

In areas such as human-computer interaction,the usability and rationality of a graphical interface determines the efficiency of the operator,so it is imperative to propose an objective and reliable evaluation method for graphical interfaces.In this paper,we not only propose a method for evaluating the graphical interface,but also propose a cognitive load state classification method based on the EEG(Electroencephalogram)signal,using the HMD(Helmet Mounted Display)interface as the research object.Our work is divided into two parts:1.A graphical interface evaluation method that combines BubbleView and ERP with greater robustness is proposed.Since most of the previous work has been done in only one way to evaluate the graphical interface,either with the help of Event Related Potentials(ERP)or eye tracking technology,the disadvantage of eye tracking technology is that it does not reveal the inner mechanisms of the brain during the cognitive evaluation process,and the equipment involved in eye tracking technology is expensive and requires calibration,the disadvantage of ERP is unable to reflect the interface evaluation process of the human eye gaze pattern.To overcome the above shortcomings,we synthetically evaluate the cognitive load of users under a graphical interface by examining the cognitive process from attentional resource input to complete attentional resource allocation.We use the P2 and P3b components of the ERP results to reflect the amount of attentional resources invested in the graphical interface and the visual importance heatmap generated by BubbleView to reflect the allocation of attentional resources.In the end,we have combined the results of ERP and BubbleView to discuss and analyze the cognitive load under different interfaces and get a more objective cognitive load evaluation result.The results show that the cognitive load of the subjects was smaller in the simpler layout of the interface.Using BubbleView's heatmap of visual importance,we also found that in the graphical user interface users are more concerned with the upper areas of the interface.2.A cognitive load classification method is proposed for graphical interface.The method uses Multi-CNN(Multi-Convolutional Neural Networks,Multi-CNN)to extract the frequency domain and spatial features of the EEG signal,then Att-BLSTM(Attention Based Bidirectional Long Short-Term Memory Network,Att-BLSTM)to extract the time domain features in the raw EEG signal,and finally constructs a cognitive load classification method by multi-feature fusion.The experimental results show that the test results of this method on the dataset are on average 82%accurate,and compared to the traditional machine learning method,this method has stronger EEG signal representation ability,and compared to other deep learning methods,it can also extract the time domain features of EEG more accurately and with stronger robustness.The results of this paper reveal the effect of different graphical interface layout on the cognitive load of the subjects,which provides theoretical basis and new ideas for designing and optimizing interfaces.Our proposed method of cognitive load classification under graphical interface can achieve accurate classification of low and high cognitive load states,which can help to realize the cognitive feedback mechanism and ultimately improve the efficiency.