Research On FMRI Visual Information Decoding Based On Multi-Voxel Pattern Analysis

Vision is the main form to obtain information from the objective world for human beings, and one of the most important means for human to understand the objective world. As an important part of brain——the most complex and sophisticated system in nature, visual system has the highest efficiency for visual information processing compared with any man-made machine. With the rapid development of neuroimaging, the appearance of functional magnetic resonance imaging(fMRI) has brought possibilities for visual information decoding and makes it gradually become a research hotspot. Visual information decoding with fMRI requires analysis of complex, noisy, multivariate data. Treating each voxel as a separate feature, conventional univariate methods often select features with redundant information or discard some informative voxels which are not significantly activated. Visual information decoding based on Multi-voxel pattern analysis(MVPA) treats the multi-voxel activation of brain as a pattern, and decodes stimuli-related information from visual cortex using pattern classification methods. Recently, MVPA has become an important way for neural coding and information processing mechanism research based on fMRI, and a critical method for visual information decoding.Based on visual information processing mechanism of human brain and the characteristics of fMRI techniques, this thesis studied the voxel selection methods, feature extraction(or dimensionality reduction) methods, visual information decoding methods and the acuity of fMRI data in the MVPA framework. The main works include:1. Study on the voxel selection methods in the MVPA framework. Conventional voxel selection methods either select features with redundant information or discard some informative voxels which are not significantly activated. Focusing on this issue, this thesis proposes a multivariate voxel selection method. This method introduces and improves the principal feature analysis(PFA) method for fMRI data processing. Each voxel is projected into low-dimensional principal component space, and in this space voxels gather to many clusters. For each cluster, only the voxel closest to the center of cluster is retained. After removing some noisy voxels, we get the final voxel set. It is validated in real fMRI data and the results show that PFA approach could retain more information with fewer voxels compared with conventional methods, thereby improving the performance of visual objects decoding.2. Study on the feature extraction(also known as dimensionality reduction) methods in the MVPA framework. Conventional methods often discard some information related to the experiment conditions when reducing dimensions, thereby impairing subsequent classification performance. To address this issue, this thesis uses "prototype examples" under different experimental conditions as a constraint. Based on the sparse representation principle of visual information, sparse learning algorithms are used to learn the transformation relationships from high-dimensional voxel space to low-dimensional feature space. The proposed method can efficiently change high-dimensional data classification with small set of examples into low-dimensional data classification with relatively larger set of examples. Compared with conventional methods, such as principal component analysis and multidimensional scaling, the proposed method could remain much more information related to experiment conditions. Thus, it significantly improved the classification performance in real fMRI data.3. Study on visual information decoding methods in the MVPA framework. Considering the characteristics of visual information processing mechanism and neuroimaging, this thesis proposes two visual information decoding methods based on the analysis of MVPA features. One applies PFA to select voxels, and uses support vector machine(SVM) to classify different conditions. The other applies the proposed dimensionality reduction method to extract features, and classifies different conditions with simple classifiers(such as linear discriminant analysis). Results in real fMRI data show that, compared with conventional methods, both methods get higher performance of visual decoding combined with proper classifier. The later method could achieve higher classification performance for linearly separable situations, while the former can be easily generalized to non-linear situations.4. Study on the acuity of fMRI-based visual information decoding in the MVPA framework. Analysis on the biological basis of voxel patterns indicates that fMRI-based visual decoding lacks high-spatial-frequency information. In one fMRI voxel, there will be up to tens of thousands of neurons. Therefore, visual information contain in voxel patterns is mainly in the macroscopic voxel-level, reflecting the low-spatial-frequency components of visual information. Towards the application of brain-machine-synergic image retrieval system, this thesis proposes a classification method that combined fMRI voxels in visual cortex with high-spatial-frequency features of images. This method merges a small amount of high-spatial-frequency information into visual voxels for visual information decoding. Analysis on real fMRI data proved the conclusion that fMRI-based visual decoding lacks high-spatial-frequency information. And the proposed method has valuable reference for practical fMRI-based image retrieval system.