Research On Key Technologies Of Image Recognition Based On RSVP-BCI Paradigm

Research on Brain-Computer Interfaces(BCI)can provide new methods of communication and control channels between the human brain and external devices,expediting further development and utilizing the potential functions of the human brain.Rapid Serial Visual Presentation(RSVP)is one of the BCI paradigms.It is an image recognition paradigm designed using a characteristic of the human brain which finds the "novel" property of the human brain for visual stimuli.Different electroencephalography(EEG)signals are evoked when subjects see new images,and image recognition can be done by decoding the relevant signals.The RSVP paradigm is suitable for processing a massive stream of image data and has broad application prospects in military target detection,medical target screening,and public opinion detection.However,the RSVP brain-computer interface paradigm development based on EEG signals still faces many challenges:(1)Due to the development of data acquisition equipment and the cost reduction of electrode raw materials,the number of electrode lead channels has increased significantly.While more channels are useful for collecting more EEG signals,some channels also produce more redundant noise and irrelevant information;(2)The target and non-target EEG samples of the original RSVP method are highly imbalanced.During the decoding process of the EEG signals,it is difficult to correctly identify the target sample,which hinders the improvement of classification performance;(3)EEG signals usually have a low signal-to-noise ratio and are unstable.The efficiency of the RSVP single-trial classification algorithm can be improved;(4)Due to the large individual differences between subjects,solving the problem of zero calibration data is hard and has poor adaptability in pairing humans with machines.In response to the problems mentioned,this study combines optimization theory and deep learning methods and focuses on a series of key technical problems in the signal decoding process of RSVP,including channel selection,data enhancement,single-trial P300 detection,and zero training samples.The main research contents and innovative achievements of the thesis are as follows:1.To solve the problems of high feature dimension of RSVP channels and weak performance in cross-subject generalization classification,this thesis proposes an RSVP channel selection model based on large-scale sparse multi-objective optimization.This method is new to the field.This method first establishes a mathematical model of RSVP channel selection and improves the convergence of the optimization algorithm by improving the generation method of the initial solution and the reduction evolution operator.Secondly,combined with the Hierarchical Discriminant Component Analysis(HDCA)method,optimization and classification are performed simultaneously to obtain the channel optimization scheme for each subject.Finally,the proposed channel combination template is validated for cross-subject generalization performance.Experimental results show that the proposed channel selection model can achieve better classification results using only 42% of the original number of channels in the public RSVP dataset.As for the problem of cross-subject generalization,the proposed combination template of the channels is better suited for RSVP than the Hoffmann empirical channel.2.To solve the problem of the imbalance in the number of RSVP target and nontarget samples,which hinders classification performance,this thesis proposes a data augmentation method to solve the class imbalance based on a conditional generative adversarial network.The method first uses an auto encoder to train the network parameters to ensure that high-quality EEG initial solutions can be generated.Then,in the generative adversarial network learning process,the labeled latent variables in the auto encoder are used to adjust the class label orientation to generate a few classes of EEG sample orientations for training.Finally,the method is verified on the selfcollected data set,and the results show that the proposed algorithm can greatly improve classification performance caused by the imbalance of RSVP categories,and can also generate artificial samples to partially replace the real RSVP data,reducing the acquisition time of EEG signals by about 40%.3.To solve the RSVP paradigm’s low single-trial EEG noise ratio,especially the low decoding efficiency of the P300 signal,this thesis proposes a P300 detection model that integrates multi-scale convolution and the attention mechanism.First,the parallel connection of convolutional layers of multi-scale receptive fields is used to enhance the ability of the network to extract and express EEG signals.Secondly,the squeeze excitation attention mechanism and the efficient channel attention mechanism are introduced to realize the extraction of important information from the P300 signal.These attention mechanism modules try to extract useful information in the temporal and spatial domains of RSVP data while suppressing useless information,which improves the classifier’s ability to detect P300 signals.In this thesis,by applying the proposed algorithm to the classification and recognition of two paradigms(RSVP and P300 speller),the proposed method can improve the detection ability of single-trial P300 signals.Compared with the current state-of-the-art EEGNet,the AUC values are increased by 3.48% and 5.68%,respectively.4.To solve the problem of poor cross-modal decoding performance of RSVP in zero training data,based on the idea of meta-learning,this thesis proposes a zerocalibration method for prototype network RSVP based on the hybrid attention mechanism.First,the network is trained using the data of known subjects,and the superimposed signal of a single subject is used as the input of the data.Then,lowdimensional features are extracted based on the network model mentioned in the previous chapter,and the prototype network is improved by combining it with the hybrid attention mechanism.With that,it has the ability to obtain the common feature of different subjects—the ERP prototype template.Finally,two loss functions of balanced metric learning and prediction labels are designed,and the distance between the ERP prototype template and the unknown subjects’ EEG is calculated for classification.Tests were carried out on public and self-collected RSVP datasets,and the results show that the proposed algorithm can improve the classification performance of RSVP without training data.In addition,after adding the data enhancement module and channel selection module,the model performance has been further improved,which provides technical support for promoting the RSVP-BCI online image recognition system.