Theoretical and experimental studies of multisensory integration as a coupled dynamical system

Perception and behavior are mediated by a widely distributed network of brain areas. Our main concern is, how do the components of the network interact in order to give us a variety of complex coordinated behavior? We first define the nodes of the network, termed functional units, as a strongly coupled ensemble of non-identical neurons and demonstrate that the dynamics of such an ensemble may be approximated by a low dimensional set of equations. The dynamics is studied in two different contexts, sensorimotor coordination and multisensory integration. First, we treat movement coupled to the environment as a driven functional unit. Our central hypothesis is that this coupling must be minimally parametric. We demonstrate the experimental validity of this hypothesis and propose a theoretical model that explains the results of our experiment. A second example of the dynamics of functional units is evident in the domain of multisensory integration. We employ a novel rhythmic multisensory paradigm designed to capture the temporal features of multisensory integration parametrically. The relevant parameters of our experiment are the inter-onset interval between pairs of rhythmically presented stimuli and the frequency of presentation. We partition the two dimensional parameter space using subjects perception of the stimulus sequence. The general features of the partitioning are modality independent suggesting that these features depend on the coupling between the unisensory subsystems. We develop a model with coupled functional units and suggest a candidate coupling scheme. In subsequent chapters we probe the neural correlates of multisensory integration using fMRI and EEG. The results of our fMRI experiment demonstrate that multisensory integration is mediated by a network consisting of primary sensory areas, inferior parietal lobule, prefrontal areas and the posterior midbrain. Different percepts lead to the recruitment of different areas and their disengagement for other percepts. In analyzing the EEG data, we first develop a mathematical framework that allows us to differentiate between sources activated for both unisensory and multisensory stimulation from those sources activated only for multisensory stimulation. Using this methodology we show that the influences of multisensory processing may be seen at an early (40--60 ms) stage of sensory processing.