Deep Neural Networks With Multi-spike Learning

As a new generation of brain-inspired neural networks,spiking neural networks(SNNs)potentially have the advantage of efficient computation by using spikes for information representation and transmission.However,due to the inherent non-linearity,discontinuity and complexity of spiking neuronal dynamics,training deep SNNs is rather challenging.A simple yet effective strategy is to construct deep SNNs by using traditional artificial neural networks(ANNs).However,how to benefit from the accuracy of ANNs and the efficiency of SNNs is still unclear.In this thesis,we first propose the augmented spikes,which utilize an additional dimension to carry the spiking intensity information.Experimental results show that Aug Mapping is faster,more accurate and efficient than other baselines.In order to break the performance bottleneck of ANN,it is more meaningful to study the direct training algorithm of deep SNNs in the long run.A new deep multi-spike learning algorithm is proposed,which identifies critical time points to construct error signals according to specific rules,and then the error signals are back-propagated to each hidden layer for updating parameters.Experimental results show that the proposed learning rule achieves good accuracy on benchmark datasets,and can effectively learn information under different codes.However,such back-propagation-based direct learning algorithm requires a layer-by-layer propagation to compute gradients,consuming massive computing resources and preventing parallelization of training.A biologically motivated scheme of local learning provides an alternative to efficiently train deep networks but often suffers a low performance of accuracy on practical tasks.In this study,we focus on a supervised local learning scheme where each layer is independently optimized with an auxiliary classifier.Accordingly,based on the way how gradients are calculated,we propose three new spike-based local learning rules,namely ELL,BELL and FELL.The ELL only considers the direct dependencies in the current time,while the BELL and FELL additionally incorporate temporal dependencies through a backward and forward process,respectively.The effectiveness and performance of our proposed methods are evaluated with the mainstream datasets.Experimental results show that the proposed method substantially outperforms other spike-based local learning algorithms.This thesis provides new effective and efficient learning algorithms for constructing deep SNNs,which will provide inspiration for understanding the spike-based information processing and learning mechanisms in the brain,and will contribute to the development of brain-like computing and artificial intelligence.