Study On The Methods For Human Visual Memory Mental Representation Modeling Based On Probabilistic Reasoning

Visual mental representation refers to the representation of an object in a person’s mind,which has been perceived in the past but not actually present to the sense.Studying the visual mental representation of human brain memory can not only deepen the understanding of human brain visual memory mechanism,but also is of great significance in application in the fields of Brain-Computer Interface(BCI),medical rehabilitation,and auxiliary case,etc.However,the existing visual memory models can only categorize the studied images during memory recall and fail to reconstruct the studied images,leading to the inability to simulate the process of visual mental representation.To this end,in this thesis,we mainly study the methods of modeling visual mental representation based on probability and Bayesian reasoning theory in the context of cognitive psychology and cognitive neuroscience,respectively.The main work of this thesis is as follows:1.The related theories and methods of visual mental representation modeling are studied and analyzed.Particularly,the free energy minimization theory,Bayesian probability reasoning theory and Bayesian Canonical Correlation Analysis(BCCA)method in human brain memory modeling are summarized,which lay the theoretical foundation for the following research work.2.A visual mental representation model based on Classification Restricted Boltzmann Machine(Class RBM)and free energy minimization is proposed.Firstly,the input layer and output layer of Class RBM are modeled to represent the original visual images and corresponding class labels respectively,while the hidden layer is used to store the encoded visual information.Then,the free energy minimization principle is applied to train the model,and the training parameters are stored.Finally,the learned visual images can be reconstructed by sampling the visual units over the hidden units with probability,which can be used to simulate visual mental representation.Experiments on the MNIST and the Face databases show that the proposed model is capable of accomplishing the visual mental imagery in one’s mind,and its performance is better than that of using the existing Restricted Boltzmann Machine(RBM)and Deep Belief Network(DBN).3.Based on functional Magnetic Resonance Imaging(f MRI)data,the neural activity representation of perceived visual images in human brain was studied,and a visual mental imagery model based on BCCA and attention mechanism is proposed.BCCA is used to build the non-linear mapping relationship between the pixels of visual images and the voxels from brain activity responses by introducing hidden variables,the Deep Neural Network(DNN)based on attention mechanism is established to encode the visual information and the Deep Generative Network(DGN)is applied to decode the visual mental imagery.Experimental results are conducted on two publicly available f MRI datasets and the results are assessed in both qualitative and quantitative analysis,showing that the proposed model is able to achieve satisfactory effects on visual mental representation simulation.