Research On Decoding Of Visual Information Of Human Brain Based On Deep Learning Models

As the most complex organ of the Central Nervous System,the brain regulates our higher neural activities such as consciousness,language,learning and memory.The Occipital Lobe located at the back of the brain is the Visual Cortex,which is the part of the cerebral cortex primarily responsible for visual information.Since human perception and understanding of the external world mainly rely on visual information in the brain,the interpretation of visual information is an important means to explore the brain's information processing mechanism,and is also a hot research topic in the field of neuroscience and information.Functional Magnetic Resonance Imaging(f MRI),a Neuroimaging technique that provides high-resolution images of brain activity while being non-invasive,is one of the main Neuroimaging techniques for studying visual cognition in the brain.Aiming at the problem of decoding the visual information of the human brain,this article mainly uses the deep learning model to decode and visualize the f MRI signal obtained by the human brain under the stimulation of the static image.Taking the f MRI data of human brain visual information as the research object,the f MRI data is first converted into network structure data to increase the amount of information,and a Graph Convolutional Network(GCN)model is established for category decoding.Then,according to the dimensional characteristics of the f MRI data,the 3D Convolutional Neural Networks(3D CNN)is used as the decoding model,and the performance of each voxel is explained in combination with visualization technology.The main tasks include:(1)A visual information decoding method based on Graph Convolutional Network model is proposed.First,the f MRI data is preprocessed,and feature selection is used to reduce noise and delete redundant features.Then,in order to increase the amount of information of the voxel,the value of the voxel itself is used as the node,and the Euclidean distance between the voxels is used as the edge to establish the graph structure data.Based on the spatial construction method of the Graph Convolutional Network the Graph Convolutional Network is established as the decoding model.The results show that the Graph Convolutional Network model has a good classification effect on the f MRI signals obtained by the human brain under the stimulation of static images,and the accuracy rate is higher than that of the traditional machine learning model.(2)A visual information decoding method based on 3D Convolutional Neural Network model is proposed.As four-dimensional data,f MRI data has both time and three-dimensional spatial information.The 3D Convolutional Neural Network can fully extract the features of the spatial domain,and accurately express and obtain the information of the voxels in the human brain's visual area.Therefore,a 3D Convolutional Neural Network model is established as a decoding model,and category decoding experiments are carried out.(3)Visualize f MRI features based on Grad-weighted Class Activation Mapping(Grad-CAM)technology.Through the target category predicted by the 3D Convolutional Neural Network model,the fusion weight of each voxel is inversely derived,and it is mapped to the neuroimaging of the human brain,and finally the human brain slice image and the three-dimensional image of the human brain are drawn for analysis.The results show that when the model distinguishes different types of objects,the voxels of the visual area have different performances.