Research On FMRI Visual Information Encoding Model Based On Deep Neural Networks

Visual information is one of the main way for human to contact with the outside world.Human brain has many advantages in dealing with complex input visual information,such as high efficiency and robustness.Human brain produces a series of sensory neural activity when exposure to external visual information.Visual information encoding based on functional magnetic resonance imaging(fMRI)mainly explores how to build a computational model to simulate this complex processes of information processing.This kind of computational model is often referred to as visual encoding model.In recent years,deep learning has gradually entered a stage of rapid development.To a certain content,the construction of deep neural network has learned from the working principles of human brain,and provide a new method for simulating human brain's visual function.The research of fMRI visual information encoding model based on deep neural networks can help to deepen human perception of visual system's working way.It is of great theoretical significance and application value both in the fields of neuroscience and artificial intelligence.In this paper,the different visual areas located in visual ventral pathway are taken as the objects.This research focus around the application of deep neural network in fMRI visual information encoding model,based on the perception characteristics of different visual areas and the advantages of deep neural networks,and devoted to the optimization of fMRI visual information encoding model based on deep neural network for different human visual cortex and image identification method based on multi-level regulated encoding model.The main work are described as follows:1.According to the receptive field characteristics of neurons in visual cortex V1,a visual information encoding model for V1 with network structure is established by the convolution neural network which can be trained end to end.A traditional visual information encoding model for V1 usually utilizes simple filters to simulate the receptive field characteristics of neurons in visual cortex V1,projects input images on to filters for convolutional operation,then transforms feature map to brain signals through linear mapping,which is consistent with the way of image information computation in Convolutional Neural Networks(CNN).CNN has strong learning ability of nonlinear approximation.It can combine nonlinear feature mapping with linear mapping,and then optimize networks in training data through end to end training.The structure of the encoding network model includes a convolutional layer and a fully-connected layer,the inputs of the network are natural images,while outputs are predictions of brain signal in V1.The network is trained by real brain signal in V1 measured by fMRI to solve the optimal receptive field characteristics of neurons in V1.The experimental results show that compared with the existing V1 visual information encoding model,the prediction accuracy of V1 brain signal has been significantly improved.2.According to the mechanism of visual information processing in the intermediate visual cortex V2,an end-to-end visual information encoding model for V2 is established.V2 is the main output area of V1 visual information,visual information it contained is complex and changeable,and the way of information processing has a strong hierarchy.There are two problems in existing visual information encoding model for V2,feature space is not rich enough and lack of hierarchical processing of different level feature information.Deep Convolutional Neural Networks(DCNN),which is similar to the human brain visual system,can effectively improve these problems by its hierarchical information processing and feature expression.An end-to-end visual information encoding network model for V2 is established by DCNN.First,the optimal CNN output layer corresponding to each V2 brain signal is found,and then encoding network model is established.The inputs of the network are natural images,while outputs are predictions of brain signal in V2.The network is trained by brain signal in V1 to solve the optimal feature expression and linear mapping.The experimental results show that compared with the existing V2 visual information encoding model,the prediction accuracy of V2 brain signal has been significantly improved.3.According to the function and information processing mode of visual ventral pathway,a natural image identification method based on multi-level regulated visual encoding model is proposed.Decoding fMRI visual signal and identifying the stimulus images seen by subjects is mainly realized by matching the real brain signals obtained by fMRI with the brain signals predicted by encoding model,then identify corresponding stimulus information from the image library.An end to end visual encoding model which is related to human visual ventral pathway is established by DCNN,and regulated by multi-level visual area.The first layer of the network is used to map the brain signal in V1,while the map of the V2 is set up by the output of the last layer.Then the two different modules of the network are optimized and regulated respectively by real brain signals of V1 and V2 areas,and a network which is more similar to the function of visual ventral pathway is obtained.Apply this model to visual signal decoding experiments to identify natural images,results show that the natural image identification method based on the deep neural networks visual encoding model is better than the other methods to further improve the recognition accuracy of the natural image in fMRI visual signal decoding experiment.