Cognitive Neuropsychological Mechanism Of Multisensory Encoding During The Working Memory Processing

Working memory is a limited-capacity system that maintaining and operating information in the mind,which is considered as the hub of high-level cognitive function such as reasoning and decision making.But little attention on working memory how encode multisensory information.The current research aimed to explore the cognitive neuropsychological mechanism on the multisensory encoding during the working memory processing whereby the multiple methods including the behavioral,ERP,and fMRI.The delay match approach was used to design the behavioral experiment.The ecological objects containing semantic properties such as pictures and sounds were applied as the experimental materials.There were a few results in the research following as:1.The calculations from behavioral study showed that reaction time of memory retrieval in the semantic congruent multisensory encoding condition was faster than unisensory conditions and the neutral stimuli pairing condtions.2.The ERP data based on the multisensory encoding during the working memory processing displayed that N1/P2 components relevant to attention appeared in the early stage(prior to 200 ms at the multisensory stimuli presenting),and multisensory integration occurred at the later stage(after 230 ms at the multisensory stimuli appearing)by the additive effect analysis.3.The attention-related brain regions were activated in the multisensory encoding stage of working memory.The additive effect analysis of BOLD showed that multisensory integration involved in a distribute brain network,including angular gyrus,right precuneus right superior frontal gyrus and anterior cingulate cortex,as well as sensory association cortices like left occipital gyrus and right superior temporal gyrus.The Bayesian causal inference suggested that these cortical areas participated the integration processing with a coordinated and hierarchical manner.4.According to the fusion of ERP and fMRI data,the N1/P2 presenting the early stage of multisensory encoding in the working memory mainly originated from the ventral attention network.And the brain activity exhibited a dynamic attribution that converging from sensory cortical areas to the higer-level neocortex in the multisensory integration.In summary,we could draw some conclusions in terms of the multiple studies.First,multisensory encoding of working memory produced memory retrieval benefit.Second,multisensory integration occurred in the later stage of working working multisensory encoding.Third,multisensory encoding of working memory involved different functional brain networks.The attention related ERP components(N1/P2)and attentional network activiation appeared in the early stage of multisensory encoding.And later stage of multisensory encoding namely integration processesing contained the activities of the distributed brain coritices.In other words,the multi-sensory information was initially processed in the sensory-specific cortex,and subsequently these information was integrated into the parietal cortex,meanwhile the prefrontal cortex played control function in the multisensory integration.Last,multisensory encoding of working memory included two relatively independent processing stage: attention appeared the early stage and integration occurred in the later stage.