Network Compression And Online Learning Algorithms Research Of Spiking Neural Networks

The spiking neural network is inspired by the structure and calculation mode of the biolog-ical brain.It realizes asynchronous calculation and information transmission based on discrete and sparse spiking signals,and can be deployed on a dedicated neuromorphic chip to run with extremely low power consumption,and has the potential to achieve efficient and intelligent computing.However,due to the limited computing and storage resources of dedicated neuro-morphic hardware chips,there are certain restrictions on the numerical precision and calcula-tion modes of the spiking neural network algorithms and models which needed to be deployed.In order to reduce the difficulty of deploying spiking neural network algorithms and models on hardware,and adapt to the real-time application scenarios of hardware chips,this article has car-ried out research from the spiking neural network compression and online learning algorithms.First,since most of the current spiking neural network weights are high-precision numbers,this causes the network model to occupy a large amount of storage resources when deployed on hardware.In response to this problem,this paper proposes a weight-threshold balance network conversion algorithm to obtain a spiking neural network with binary weight connections,which significantly reduces storage resource consumption and realizes the compression of the spiking neural network.In addition,online learning algorithms are usually deployed on neuromorphic chips to complete real-time learning in actual scenarios.However,most online learning algo-rithms are difficult to complete complex spatiotemporal data learning tasks,and spiking neural network algorithms that can complete this task often do not have real-time learning functions.Therefore,in order to complete the complex spatiotemporal data learning tasks in real-time ap-plication,to achieve a learning algorithm that can better meet the hardware constraints and be more bio-interpretable,this paper proposes an asymmetric spatio-temporal online learning al-gorithm.The recognition performance on image and sound datasets of the proposed algorithm is comparable to the latest off-line learning algorithm of spiking neural network and better than the existing online learning algorithm.Specifically,the main research content and contributions of this article are described as follows:·The weight-threshold balance network conversion algorithm:The spiking neural network model with high-precision weight will consume a lot of storage resources during hardware deployment,this paper studies the conversion process of existing network conversion al-gorithms,and analyzes the constrained relationship between the weight and threshold of the spiking neuron obtained by conversion,and based on the constraint,a weight-threshold balance network conversion algorithm is proposed.The weight-threshold bal-ance network conversion algorithm can scale the high-precision weight to low-precision by changing the threshold value storage precision of the spiking neuron,thereby effec-tively obtaining the spiking neural network with binary weight,which significantly re-duces the storage resource consumption and realizes the spiking neural network compres-sion.The experimental results show that compared with the high-precision spiking neural network,the binary spiking neural network can save up to 86%of storage resources.On MNIST,CIFAR-10 and CIFAR-100 datasets,the best classification accuracy of the binary spiking neural network is 99.43%,90.19%and 62.02%respectively,which is comparable to the latest high-precision spiking neural network.In addition,on the premise of hav-ing the same network structure,compared with high-precision spiking neural networks,binary spiking neural networks have better convergence.·The asymmetric spatio-temporal online learning algorithm:At present,most of the spik-ing neural network online learning algorithms are usually applied to small data sets,and are difficult to learn complex spatio-temporal input data streams.While the existing high-efficiency spatio-temporal learning algorithms of spiking neural networks usually do not have online learning functions,and the precise symmetry constraints required in the pro-cess of reversely weights update of the network are not only difficult for hardware imple-mentation,but also not biologically feasible.Therefore,in order to efficiently complete the complex spatio-temporal data learning tasks in real-time scenarios,and reduce the hardware implementation difficulty and improve the biological feasibility of the algo-rithm,this paper proposes an asymmetric spatiotemporal online learning algorithm for the multi-layer spiking neural networks' real-time training.The synapse weight update of this algorithm is only related to the internal state of the presynaptic and postsynaptic spiking neuron,and without the need for precise and symmetrical weight back propa-gation,the spatio-temporal characteristics of the input data can be effectively learned by only iteratively updating the internal variables of the spiking neuron.The classifica-tion accuracy of this algorithm on the MNIST image data set based on rate coding and the MedleyDB musical instrument data set based on time coding reached 98.23%and 95.38%,respectively.Its performance is better than existing online learning algorithms,and is comparable to the latest offline.In addition,this article also explores the impact of algorithm hyperparameters ????c on performance.The experimental results show that the smaller the threshold ? of the spiking neuron,the better the algorithm performance.