Research On Deep Network Models For Processing Brain Signals In Multiple Modalities

The technique for processing brain signals in multiple modalities is one of the most popular topics in the studies on brain science and brain-inspired computing.Based on the technique for processing brain signals in multiple modalities,we are able to study the functions in different brain regions,explore the essence in high-level human cognition behaviours,analyse the moving and memory abilities,aid the diagnosis and treatment of brain disease and so on.The technique for processing brain signals in multiple modalities could be considered as a new key tool for brain-computer interface,brain-inspired computing and brain science,which is significant in the areas like neuroscience,psychology,neuropathology and neural engineering.However,the existing techniques for processing brain signals perform poorly in feature extraction,model fitting,model training and model efficiency that more advanced processing techniques for brain signals are desired.To solve the problems and challenges in the research of brain signal processing,we introduce the deep network in this thesis.According to the characteristics of brain signals in different modalities,we improve the structures of the existing models,and propose some new deep networks.The proposed deep networks are able to efficiently process brain signals in several modalities,and could be considered as novel techniques for the research of brain-computer interface,brain-like computing and brain science.In this thesis,we proposed several deep network models for processing brain signals in different modalities,the main contents and results are drawn as follows:1.EEG-based event-related potential detection: To detect and analyse the eventrelated potential in electroencephalogram(EEG)signals,we propose two networks which are a hybrid network based on restricted boltzmann machine(ERP-NET)and spatialtemporal discriminative restricted boltzmann machine(ST-DRBM).The ERP-NET is able to extract spatial and temporal features of event-related potential signals which is helpful for the detection of the signals.The ST-DRBM is able to extract discriminative spatial and temporal features which could accurately reflect the spatial distribution and the change with time in the event-related potential signals.Therefore,the ST-DRBM can effectively detect the event-related potential signals.According to the experimental results,the two networks could achieve high detection performance on event-related potentials.2.MEG signal decoding: The research on brain signal decoding is a popular topic in brain science recently.The brain signal decoding technique is significant for studying highlevel human cognition behaviours,analyse emotional changes,diagnose mental diseases and so on.In this thesis,we propose a recurrent neural network based on gated recurrent units,and apply for decoding the brain magnetoencephalogram(MEG)signals evoked by face images.The proposed recurrent network could effectively extract the features of MEG signals in different subjects,and decode the signals across subjects.In the crossvalidation experiments,the proposed network achieves a high performance in decoding the MEG signals.3.Brain tissue segmentation in magnetic resonance imaging(MRI): Based on the segmentation results of brain tissues in MRI data,we are able to quantify the structural capacity of the brain,assess the neurological health status,and diagnose brain diseases like alzheimer’s disease,dementia,focal epilepsy,Parkinson’s disease and multiple sclerosis.In this thesis,we propose a multi-modality aggregation network which is able to extract multi-scale features of brain MRI images.Based on multiple subnetworks,the proposed network could effectively utilize the complementary information of MRI images in three different modalities,which is helpful for more accurate segmentation of brain tissues.The experiments on the database of brain segmentation competition(MRBrainS Challenge)demonstrated the abilities of the proposed network.The proposed multimodality aggregation network outperformed most of the existing models,got the first place in history and the second place in the current ranking.In conclusion,we propose several deep networks in this thesis.The proposed networks achieved high performance in the tasks of EEG-based event-related potential detection,MEG signal decoding and brain tissue segmentation in MRI.Our work may provide a new idea and methodology for the researchers on brain signals,which are promising for the research and applications in brain-computer interface,brain-inspired computing and brain science research.