Design And Simulation Of Weight Updating Algorithm In Memristor Neural Networks

Artificial intelligence technology is deeply influencing the contemporary society.With the major breakthrough in the filed of machine vision and speech recognition,neural network has become the key technology scheme of modern artificial intelligence.However,the artificial intelligence algorithms running on the digital platforms based on traditional silicon chips are largely restricted by the nearly ending of Moore's Law and traditional von Neumann architecture,leading a poor energy efficiency(performance-power ratio)of the scheme.Therefore,there is an urgent need for a new neuromorphic computing architecture beyond CMOS technology.In recent years,memristor(a kind of resistance device with memory function)has become a new hardware solution of neuromorphic computing due to its fast and efficient matrix vector operation.Attribute to its great energy efficiency,the neuromorphic computing based on memristor has also attracted widespread attention.For the inference process of neural networks(forward propagation),memristors can be regarded as variable resistors with tunable synaptic weights(conductance).Vector multiplication can be easily realized by using Ohm's law and Kirchhoff's law.For the training process of neural networks(backward propagation),the weight to be updated can be calculated according to the stochastic gradient descent algorithm(SGD),and the synaptic weight can be updated by applying corresponding number of electrical pulses to the device terminal.However,the nonlinear weight updating property of memristor makes it difficult to be trained in neural network learning process,which leads a poor performance of the memristor neural network.In this paper,we modify the differential circuit structure of the memristor array,and develop a self-adaptive learning method based on this circuit structure for the weight updating,in which the error caused by the memristor nonlinearity can be effectively suppressed and the overall performance of the network can be greatly improved.In addition,the weight updating method proposed in this paper has good hardware friendliness.In the process of weight updating,there is no need to read the precise conductance value of the memristor and calculate the specific number of electrical pulses applied to the device terminal.Compared to the traditional nonlinear stochastic gradient descent(SGD)updating algorithm and the piecewise linear(PL)method which are most often used in memristor neural networks,the method proposed in this paper can greatly reduce the complexity of peripheral control circuit.Finally,we construct the classical convolutional neural network LeNet-5 based on memristor,and complete the weight update process of LeNet-5 according to the method proposed in this paper.The simulation results show that the recognition accuracy of the proposed scheme for Modified National Institute of Standards and Technology handwriting digits datasets can reach nearly 94% under the typical case that memristor nonlinearity is±1.Its online learning performance is significantly exceed the traditional nonlinear SGD algorithm based on memristor network.