Unsupervised Learning Algorithm Based On Kullback-Leibler Divergence For Deep Spiking Neural Networks

With the rapid development of big data technology,the research and develop-ment of brain-like computing models and platforms has attracted more and more attention of researchers.Deep Neural Network(DNN)plays an important role in various industries by its excellent processing ability for large-scale data.However,there are some gaps between the traditional artificial neural network model and the biological nervous system in information coding,which makes the DNN compu-ting model with bio interpretability need further research and development.Spik-ing Neural Network(SNN)uses precise timing spike sequence coding and pro-cessing neural information.Compared with traditional artificial neural network based on spike rate coding information,it has more powerful computing power and are very suitable for implementing complex space-time mode processing problems.In addition,in the training of neural network model,unsupervised learning does not need to label the data,and unsupervised learning is more in line with the char-acteristics of brain learning.Therefore,it will be a challenging task in machine learning to construct unsupervised learning method of Deep Spiking Neural Net-work(DSNN)by combining the information processing mode of SNN with the calculation mode of DNN.Based on the above description,this thesis constructs an unsupervised learning algorithm for DSNN.The main research contents are as fol-lows:(1)The unsupervised learning rules and deep learning methods of SNN are analyzed.and then the traditional Contrast Divergence(CD)algorithm is intro-duced.On this basis,the CD learning algorithm of SNN is analyzed,which includs the CD learning algorithm based on spike rate and the CD learning algorithm based on STDP mechanism.(2)For SNN with spike sequence coding,an unsupervised learning algorithm based on KL divergence is proposed.Firstly,the spike neuron model used in this thesis is introduced,and the Spike Neural Machine module constituting the deep spike network is constructed.Then,the unsupervised learning algorithm is used in the spike sequence learning task to verify the learning performance of unsuper-vised learning rules by the reconstruction error.Finally,the experimental results are analyzed.The results show that the proposed unsupervised learning algorithm can learn spiking sequences with spatiotemporal characteristics and has good learning performance.(3)The deep structure of the SNN is designed by module superposition,and the corresponding layer-by-layer pre-training method is proposed.In order to veri-ly the computational performance of the deep structure of the SNN,this thesis ap-plies the computational model to image classification.Firstly,features are learned by unsupervised layer-by-layer pre-training,and then a supervised learning algo-rithm is added to the last layer to fine-tuning the network.In this thesis,the error back propagation algorithm only fine-tunes the synaptic weights of the last hidden layer and does not back propagation the whole network.The experimental results show that the application of unsupervised learning algorithm based on lay-er-by-layer pre-training and back fine-tuning method can solve the problem of im-age classification well,and get a higher accuracy.