Simulation And Evolution Of Large-scale Spiking Neural Networks

During last few years we have witnessed a shift of the emphasis in the artificialneural network community toward spiking neural networks. Motivated by neurosciencediscoveries, many studies consider large-scale neural networks with spike-timing as anessential component in information processing by the brain. While the study of spikingneural networks as a neuroscience research methodology still faces difficult conceptualand technical challenges, it is a promising and timely endeavor. This thesis motivates toaddress two fundamental issues which are: (1) how spiking dynamics of each neuron andnetworks of spiking neurons are simulated; (2) how large-scale spiking neural networksare developed and evolved.Combining the dynamical property of Hodgkin-Huxley neuron model and the ana-lytical property of integrate-and-fire neuron model, we propose a novel two-dimensionalpiecewise linear spiking neuron model. We show that this framework allows a qualita-tive description of excitable systems through bifurcation theory but also a quantitativeanalysis of neuronal behavior through an explicit analytical representation of the statevariables. A detailed analytical study of the model is presented. The model gives riseto new neuro-computational properties not present in one-dimensional integrate-and-fireneuron models. In experiments, using this model we simulated the spiking and burstingbehavior of known types of cortical neurons.Neural computation relies heavily on the simulation of large-scale networks of neu-ron models. We propose a novel integrate-and-fire neuron model with exponential synap-tic conductances that can be simulated exactly. The postsynaptic potentials and sponta-neous discharge statistics of the new model are compared with those of commonly usedmodels, such as the leaky integrate-and-fire model with instantaneous synaptic interac-tions or the passive membrane equation model with exponential synaptic conductancesin which conductances are explicitly integrated. The proposed model is much closer tothe passive membrane equation model with respect to the spiking response dynamics,while still being nearly as computationally efficient as simple leaky integrate-and-firemodel. Then we present an event-driven simulation strategy for the new model. Usingevent-driven and clock-driven simulation strategies we simulate large-scale random net-works, the results show that (1) the simulation time scales linearly with the total numberof spiking events in the event-driven simulation strategies and (2) the temporal precision of spiking events impacts on neuronal dynamics of network in the different simulationstrategies.Based on a model of network encoding and dynamics called the artificial genome,we propose the fixed-size and variable-size segmental duplication and divergence mod-els for evolving genetic regulatory networks. Using analytic and simulation techniques,we find that the two classes of networks share structural properties with natural tran-scriptional regulatory networks. Specifically, these networks can display scale-free andsmall-world structures. We also find that these networks have higher probability to op-erate in the ordered regime, and lower probability to operate in the chaotic regime. Thatis, the dynamics of these networks is similar to that of natural networks. The resultsshow that the structure and dynamics inherent in natural networks may be in part due totheir method of generation rather than being exclusively shaped by subsequent evolutionunder natural selection.There is a growing recognition that natural selection is not the sole determinant ofthe direction of evolutionary change. Recent work has attempted to show the importanceof developmental bias in shaping organic form. We propose a computational model of adevelopmental system of artificial cell lineages. We use the model to generate randomorganisms with different levels of phenotypic complexity and analyze mutation opera-tors for their effects on developmental bias. We find that developmental bias exists in themodel system and varies with the model parameter. We also show that developmentalbias varies strongly with phenotypic complexity and mutation operator. Finally, we il-lustrate how developmental bias affects the direction of evolutionary modification fromthe manner in which accumulated mutations cause an increase in phenotypic complexity.These results suggest that mutation operators could have strong in?uences on develop-mental bias.Research in evolutionary neural networks is inspired by the evolution of biologicalbrains. Because natural evolution discovered intelligent brains with billions of neuronsand trillions of connections, perhaps evolutionary neural networks can do the same. Us-ing the artificial genome model as a framework for describing genetic regulatory net-works, the dynamics of gene expression can be treated as a model for cell fate speci-fication. We propose a developmental method for evolving large-scale spiking neuralnetworks. The advantage of this method is that it can facilitate fast and efficient de-velopment of spiking neurons, neural connections, and synaptic plasticities. The cor-responding evolutionary experiment shows that the intelligent behavior emergences forthe neurally-driven autonomous agents in a food gathering task. Additionally, it also shows that due to the efficiency of the proposed method, large-scale spiking neural net-works can be easily managed thereby making it suitable for long durational evolutionaryexperiments.