Research On Stochastic Computing Method Of Deep Neural Network

Artificial intelligence technology represented by deep neural network is widely used in many fields such as object recognition,image classification,natural language processing and automatic driving,bringing great improvement to the production and life of the society.However,due to the high computational complexity and high storage requirements of the deep neural network model,the deep neural network has encountered great challenges in the process of embedded device deployment.In order to solve the contradiction between the limitation of embedded hardware resources and the high resource demand of deep neural network,a kind of stochastic computing method which is different from the traditional binary computing method has attracted more and more attention.Compared with binary computing,stochastic computing has the advantages of smaller hardware area,lower power consumption,shorter critical path and stronger fault tolerance.However,due to the characteristics of probability and redundancy of stochastic computing,the improvement of computational accuracy often leads to the large increase of computational delay and energy consumption,which is contrary to the original intention of stochastic computing.To solve this problem,this paper focuses on the circuit design of stochastic computing in deep neural network,and explores the improvement of stochastic computing in neural network from three aspects: coding optimization of stochastic computing,design of high-precision computing module and optimization of overall architecture.The contributions of this paper mainly include:Firstly,the influence of two deterministic sequence coding methods on the accuracy of stochastic computing was evaluated,and the appropriate coding combination method was selected and applied to the Inner-Product module.Experimental results show that compared with LFSR coded sequence,the sequence with deterministic coding can improve the accuracy of stochastic computing.Secondly,a high precision multi-input Stochastic Approximate Counter(SPC)is proposed.The experimental results show that the computational accuracy of SPC is better than that of the current Approximate Parallel Counter(Ax PC).Combined with the sequence coding optimization method,the computational performance of the InnerProduct based on SPC was improved greatly.Thirdly,two feature extraction block architectures were designed,and the reasoning accuracy of the two architectures on MNIST data set was evaluated experimentally.Experimental results show that when the sequence length is 128 bits,the overall reasoning accuracy of the mixed random architecture decreases by only 0.58%.Compared with the CPU,the Le Net-5 model reasoning process based on stochastic computing reduces the power consumption by 50×,improves the throughput by 297×,and achieves the energy efficiency of 63620 Images/J.