Research On Spiking Neural Network With Respect To The Time-series Pattern

In real world scenarios,datasets can be classified into static and timeseries datasets.For example the multi-channel Bio Electrical singals,kinematic joint motion datsets,spatio-temporal traffic flows and audio signals are all time-series datasets.Different from static images,in which the pixels are kept high independency between the ‘background' and ‘subject',the local information among the time stamps is correlated.In previous studies,artificial intelligence methods were used to model the practical tasks based on time-series.However,the learning efficiencies of these models were not very well,especially meeting with long-term prediction task of complex time-series datasets.Besides that,their generalization performances for deep learning model were always limited.In this dissertation,we focus on the research of complex time series,and the corresponding contributions include:1.The probability-modulated spiking neural network is proposed to model the spatial-temporal time-series datasets.The probability-modulated spiking neural network shows its advantage when the spiking units keep multiple spike mechanism.For traditional neural network models,considering of the theoretical difficulty of the discontinuity of the membrane potential,standard gradient descent optimization cannot be applied directly.In this dissertation,the Bayesian mechanism is introduced in probability-modulated spiking neural network to calculate the firing probability of the spike trains at each time stamp.The discrete membrane potential of each spiking unit can be converted to the continuous firing probability.When dealing with complex spatiotemporal datasets,the probability-modulated spiking neural network combines Gabor filter,Max Competition Layer and asynchronous motion detector to form the learning framework.The probabilitymodulated spiking neural network is compared with the state-of-art models on the common benchmarks.The validation for the experiments show the probability-modulated spiking neural network achieves fewer learning parameters and higher learning efficiency than state-of-art models.2.The monitor-based recurrent spiking neural network is proposed to predict long-term complex time series datasets.For the prediction of the time-series,we regard the spiking neural network as a dynamic system and “monitor” is embedded into the spiking unit in reservoir to online track the components of the state space.So that the monitor-based recurrent spiking neural network can restore and predict the complex time-series.Through the validation of several relevant experiments,the results show the monitor-based recurrent spiking neural network not only achieves a high degree of the fitness,but also maintains a high spiking efficiency and memory storage.3.The Spiking Auto Encoder model is proposed to represent various types of time series datasets.For univariate,multivariate and multi-channel time-series datasets,we generalize spiking neural network into a deep auto encoder model.In addition,in order to extract the features in time domain,multi-scale reservoirs and bidirectional connection mechanism between the spiking units are employed.The advantages of the Spiking Auto Encoder have been proved through the contractivity analyzation.Respect to recognize the multi-channel time-series dataset,several spiking encoders are fused and a convolutional neural network is used as the readout part of Spiking Auto Encoder model.Comparing with the state-of-art models,the experimental results show the Spiking Auto Encoder model achieves the better performance in various types of the time-series classification.4.The deep ensemble model,MBPEP,is proposed to deal with time-series uncertainty prediction tasks.The traditional artificial intelligence models are applicable when the distribution of the training datasets and test datasets are kept same.It is a difficult problem of designing a highquality model to learn the uncertainty distribution of the test datasets.In this dissertation,the traditional deep learning models are used as the base learner.And the ensemble deep model “MBPEP” is constructed to solve the uncertainty prediction task.Through some of the uncertainty classification and regression experiments based on time-series,the results show the MBPEP model improving the effect of the uncertain prediction.5.The margin-based Pareto ensemble pruning model is proposed to search the ensemble pool size.In order to enhance the generalization ability and model's other performances,we study the decision boundary of single base learner.Based on the traditional machine learning model,margin-based Pareto ensemble pruning model are used with Pareto optimization evolutionary algorithm to global research the ensemble pool size.And the local information between the base learners are searched by margin criterion technique(MCP).Through some relevant experiments,the results show that margin-based Pareto ensemble pruning model can improve the generalization ability within small ensemble pool size.6.Based on heterogeneous programming,the accelerated spiking neural network is implemented.Facing massive and high-dimensional spatiotemporal learning datasets,the learning efficiency of spiking neural network needs to be considered.Since each spiking unit in network produces spike trains in parallel,we design an optimization model for spiking neural network based on multiple heterogeneous programming,such as graphics processing unit(GPU)and central processing unit(CPU),to enhance learning efficiency.The model is benchmarked on UCI data sets.The results show when the number of the configurations for the spiking neural network is larger,the improvement of learning efficiency of the spiking neural network is more obvious.