Design Of Spiking Neural Network Analog Circuit Based On Reinforcement Learning

With the advent of the information age,the demand for ultra-large data calculations in modern society is increasing.The traditional von Neumann computer architecture processes information in a serial manner.In many cases,it is difficult to meet the current needs.Brain-like computing has also become a research hotspot.Brain-like computing essentially simulates and draws on the structure of biological neural networks.Spiking neural networks have become the focus of research on brainlike computing because their network structures are similar to biological neural networks.As an important part of brain-like computing,spiking neural networks modify weights mainly by the time difference between pre-and post-synaptic pulses,namely the STDP(Spike Timing-Dependent Plasticity)rule.This learning mechanism works well in feature classification,but like any unsupervised learning algorithm,it does not work well in functions such as decision making and diagnosis.In biological neural networks,the brain's reward system plays a vital role in judgment and decision-making,and the reward process is also known as reinforcement learning.Therefore,combining reinforcement learning with spiking neural networks and adding a reward mechanism to spiking neural networks has resulted in a new learning rule,R-STDP,which has also become an important content of impulsive neural network research.At present,the research on spiking neural networks is not deep enough,and the design of spiking neural networks by hardware is not much,and most of the spiking neural networks that have been implemented are relatively single in application and not universal.In order to conduct a more comprehensive study of the hardware aspects of spiking neural networks,this paper has completed the design of spiking neural network analog circuits based on reinforcement learning and verified its function.The main work and results of this article are as follows:(1)A synaptic circuit based on CMOS analog circuits is designed in this paper.It mainly includes the initial weight randomization module and R-STDP-based synaptic plasticity module.The initial weight randomization module increases the firing interval between neurons,avoiding greedy learning and repeated learning due to lateral inhibition failure;the synaptic plasticity module introduces a reward mechanism for reinforcement learning in pulsed neural networks.Modulation of weights through reward signals improves the versatility of pulsed neural networks.(2)Based on the leakage integral ignition model,this paper has completed the design of a multifunctional CMOS neuron circuit,which reproduces the characteristics of threshold adaptation and refractory period in biological neurons.In addition,the weighted summation module is used to map excitatory synapses and inhibitory synapses into positive and negative weights.The two synapses are combined into one,which reduces the hardware overhead for the entire impulsive neural network structure.(3)This paper uses the above-mentioned synapse and neuron circuit model to complete the design of the impulse neural network circuit,and applies the network to the detection of XOR operation.