Circuit Design Of Memristor-based Neural Networks With In-situ Learning

Great progress has been made in the research of artificial neural networks(ANNs)in decades.ANNs are widely used in fields of image processing,speech recognition,natural language processing,industrial control,etc.With society stepping into the era of big data,more and more data need to be processed by ANNs,which requires more and more computing resources.However,existing computing systems based on the von Neumann structure are insufficient to meet this challenge.Therefore,a new computing architecture with capabilities of real-time data processing and large-scale parallel computing is urgently needed.The memristor provides an opportunity to solve this problem.The memristor is very suitable for simulating biological synapse due to its plasticity and will greatly simplify the design of the synaptic circuit.The memristor also has the advantages of low power consumption,in-memory computing,and can be easily integrated to achieve large-scale parallel computing.Based on the characteristics of the memristor and the algorithms of ANNs,this dissertation designs a memristor array with positive,negative,zero weights and nonlinear activation functions,and a weight update scheme that can in-situ update the memristor conductance is proposed.The in-situ learning can make the memristor-based neural network adapt to the characteristics of the hardware circuit in the learning process,so it has a better learning ability.And then memristor-based ANNs with in-situ learning ability,including memristor-based multi layer perceptron(MMLP),memristor-based convolutional neural network(MCNN),memristor-based long short term memory(MLSTM),memristor-based spatial pooler(MSP),are designed,and they can be applied to pattern classification tasks.The main research results and innovation achievements of this dissertation are as followsA memristor array for the construction of memristor-based neural network circuits is designed.Positive,negative,and zero weight values are realized in the array,and nonlinear activation functions are realized at the same time.An in-situ learning strategy based on the“half-voltage selection” and the two-step update is put forward.The weight update scheme does not need the control terminal,such as that in the 1T1M(One Transistor One Memristor)synapse,to control the writing of the memristor conductance.It updates the memristor conductance in parallel just by applying the row and column voltages,and is suitable for the synapses without transistors such as 1M(One Memristor)and 2M1R(Two Memristors One Resistor)synapses.Based on the presented memristor array and the weight update scheme,two kinds of memristor-based feedforward neural networks,which are MMLP and MCNN,are designed.MMLP and MCNN are trained in-situ by the proposed in-situ update scheme,solving the problems that the ex-situ learning cannot adapt to the characteristics of hardware circuit and some in-situ learning methods cannot update the memristor conductance value in parallel.The effectiveness of the networks is substantiated by pattern classification tasks.A kind of memristor-based recurrent neural network(RNN),which is MLSTM,is designed.Compared with MMLP and MCNN,MLSTM considers the time correlation of input data,so it has the ability of processing sequence data.MLSTM is also trained in-situ through the weight update scheme.Compared with existing memristor-based LSTM circuits,MLSTM has more complete functions and stronger learning ability.MLSTM is substantiated effective in processing sequential data through simulation experiments on pattern classification datasets.MSP,which mimics the neocortex,is designed based on the memristor array.MSP is trained through the competitive Hebbian learning rule.Though the Hebbian learning rule is not as precise as the back propagation(BP)algorithm,it has advantages of high speed,simple circuit implementation and low power consumption.MSP solves many shortcomings in previous spatial pooler(SP)circuits,such as the incompleteness of the realization of the functions of the SP synapse,the lack of parallel updating of synapses,and the lack of learning circuit.MSP is trained in-situ and the synapses are updated parallelly.The validity of MSP is substantiated through statistical metrics and pattern classification tasks.Finally,the work of this dissertation is summarized,research prospects for the future work are put forward.The research results of this dissertation provide ideas and references for the research and designs of various types of memristor-based ANNs.