Application Research On Memristor-based Stochastic Neural Network

By adding stochasticity to weights or neurons,the stochastic neural network can effectively overcome many challenges faced by conventional neural networks,such as overfitting,overconfident about output.Stochastic neural networks have been applied in the field of high security and reliability requirements.However,with the rapid development of artificial intelligence and the rapid increase of data scale,the traditional"separation of storage and computing " computation architecture based on CMOS device has been unable to efficiently support the core matrix-vector product operation of neural networks and massive data processing.In addition,when stochastic neural network is deployed on traditional CMOS hardware platform,the complicated random generation generator is often needed to provide the necessary stochasticity for the model.As the dimensions of CMOS devices are about to reach the physical limit,the traditional hardware platform has been unable to satisfy the demand of artificial intelligence tasks for computing power.In order to optimize the hardware platform supporting stochastic neural networks,it is necessary to develop new principal devices with stochasticity and non-volatile.Memristor,with the advantages of nonvolatile,simple structure,low power consumption,good miniaturization,easy three-dimensional integration,and compatibility with CMOS process,supports in-memory computing,parallel computing,and analog computing.It is considered as one of the promising candidates to break through the traditional computing structure and implement the new "integration of storage and computing" computation architecture.In addition,the abundant intrinsic stochasticity of memristor makes it a stochastic memory device,and these intrinsic stochasticity are very suitable for the realization of memristor based stochastic neural networkIn this thesis,we focus on how to use the intrinsic stochasticity of resistive switching devices to implement memristor-based stochastic neural network,including the test and characterization of intrinsic stochasticity of devices,the construction of stochastic synapses and neurons based on memristor,circuit design,hardware system demonstration and so on.The following achievements have been implemented:(1)Research on memristor-based Hopfield networkFirstly,we fabricated 1Kb 1T1R TiN/TaOx/HfOx/TiN device array,and measured the stochasticity of the resistance of the device,and performed the Gauss hypothesis testing.On the basis of theoretical analysis,we adopt software simulation method to study the influence of stochasticity in weight on memristive Hopfield network in solving optimization problems.The results show that there is an appropriate stochasticity level maximizing the gain of the memristive system.(2)Research on memristor-based Bayesian neural networkFifteen TiN/TaOx/HfOx/TiN devices in the 1Kb array were randomly selected,and the stochasticity of each device under different resistance states was statistically measured,and the logarithm of measurement results was fitted by a linear function.According to the measurement and fitting results,we adopt the method of re-parameterization,and use memristor array to represent any Gaussian random variable in a limited range.On this basis,we build a Bayesian neural network hardware system based on memristor.The memristive hardware system supports two operation modes:Bayesian neural network and Gaussian random number generation.(3)Research on memristor-based restricted Boltzmann machineBased on the statistical measurement of threshold voltage and holding voltage of NbOx devices,the realization of binary stochastic neuron by using the stochasticity of switching voltage of NbOx devices is theoretically analyzed.A hybrid CMOS-memristor stochastic neuron circuit is designed,and the influence of the difference of switching voltage between devices on the neuron circuit is discussed.In view of the device-to-device variation of the switching voltage,we propose a simple calibration scheme to overcome the influence of the variation on the neuron circuit.Based on the analysis and system simulation of stochastic neuron circuit,we adopt a brand-new weight mapping scheme and design a restricted Boltzmann machine hardware system based on memristor.In the memristive restricted Boltzmann machine system,the weight implemented by 1T1R TaOx/HfOx device,and the neuron implemented by the stochastic neuron circuit based on NbOx device.Due to the stochastic behavior of weights and neuron circuits in the memristive hardware system,the interaction between them is further analyzed,and the corresponding formula is derived to quantitatively analyze the interaction between them.