Research On Cross-modal Brain Connectivity Graph Generation Based On Generative Adversarial Network

The human brain is the most advanced part of the human nervous system.The information exchange and functional coordination between different brain regions are essential reasons for the advanced cognitive ability of human beings.Therefore,understanding brain function has become an important task on the way to explore the working mechanism of the brain and diagnose various brain diseases.However,the function of the human nervous system is strongly constrained by its basic structure and anatomy,and exploring the structure-function relationship of the brain has become an important means of studying brain function.In the current research,the structure-function relationship of the nervous system is increasingly conceptualized as the relationship between structure and functional brain connectivity networks,and this raises the question of generating functional brain networks through structural brain networks to generate missing functional modality brain connectivity is used for further brain function studies and classification of disease diagnosis.However,the existing methods for generating structure-function brain connectivity still have some limitations.Traditional methods based on biophysics and statistics may ignore some key neurophysiological processes and higher-order interactions.The accuracy of deep learning-based methods is high,but usually only a single brain partition template is used in these methods,confining brain connectivity network features to a single scale space and ignoring the multi-scale spatial information of brain connectivity networks.In this thesis,based on the generative adversarial network framework,a recurrent multi-scale graph generative adversarial network model combined with the multi-scale characteristics of the brain connection network is proposed for the structure-function coupling problem.For the structural brain connectivity network,the model learns its mapping with the functional brain connectivity network to generate missing functional brain network modal data.This thesis will discuss the three aspects of pre-data preprocessing,model framework design and experimental results verification.First,the original DTI and f MRI data were preprocessed,and the automatic anatomical label template was used to map brain regions to construct a set of structural and functional brain connectivity topology connectivity.Secondly,the constructed structural brain connectivity network was used as the input of the model proposed in this thesis.The multi-scale topological feature of brain connectivity is extracted,and decoding operations reconstruct functional brain connectivity.At the same time,the generation performance of the model is improved by generating adversarial and recurrent structures.Finally,we use human connections in the real dataset.Experiments were carried out on 1043 healthy human data samples for the group project.The ten-fold cross-validation was used to evaluate the similarity between the model-generated results and the real brain connections.Compared with other methods,our model obtained the best model performance.In terms of model method,this thesis proposes a recurrent multi-scale graph generative adversarial network model for the multi-scale spatial characteristics of brain connection network.The model combines medical prior anatomical knowledge and uses an anatomical prior graph pooling layer to pool the brain connection network to obtain the brain connection map structure at different scales,and use the multi-scale graph convolution layer to extract different scales.Finally,a feature pyramid graph decoder module is proposed to better integrate the semantic information of brain connectivity features at different scales for decoding,and reconstructing the brain connectivity graph structure of missing modalities.At the same time,combined with the generative adversarial network structure,the model constructs a multi-scale graph fusion discriminator to fuse and discriminate the functional brain connection networks of different scales generated by the generator,and obtain the overall discriminant result.On this basis,the model incorporates the recurrent structure to further improve the performance of the model and strengthen the correlation of different modal brain connections between a single sample.