Research On Software And Hardware Codesign Of Sequence Model Based On Neural Turing Machine

Nowadays,deep learning has been widely used in various fields,and sequence models based on recurrent neural networks have been continuously improved and innovated in speech recognition,language processing and other fields.At present,in order to improve the accuracy of the sequence model,most studies use deeper network models to improve the data fitting ability of the model through more weight parameters.The huge amount of calculation brought by this method reduces the speed of network reasoning.At the same time,this kind of network is more prone to overfitting in the application of long sequences.In order to solve the problem of slow inference speed of neural networks,academia mainly uses algorithms such as compression and distillation.Although these methods will achieve certain results,they will significantly reduce the accuracy of the network model.Neural Turing Machine can increase the network's memory and learning ability of sequences.This article uses Neural Turing Machine to improve sequence model,which not only reduces the number of network parameters,but also improves the accuracy of the network model.In order to solve the problem of slow inference speed,many practical applications currently use hardware acceleration methods.This paper proposes a hardware-based parallelization acceleration scheme based on the characteristics of the sequence model.In order to improve the accuracy of the sequence generation model,this article combines the recurrent neural network and the Neural Turing Machine mechanism.On the basis of the encoder-decoder model,an external memory module is added,so that the sequence generation model can not only compare the sequence at different moments in the encoding stage.Input produces attention effect,and can also form memory in the decoding stage,thereby improving the model effect.In order to reduce the weight of the model,this paper proposes a sequence model based on the Neural Turing Machine to share the encoder and decoder weights,which ensures accuracy and reduces the number of network weights.Aiming at the sequence classification model,this paper proposes a new sequence classification network architecture that uses a recurrent neural network combined with a Neural Turing Machine.In the process of training and inference,the Neural Turing Machine is used to record historical information.Through the addressing process of the Neural Turing Machine,the network is inferred.During the process,different levels of attention to important data in the input sequence are increased to improve the model.The complexity of the recurrent neural network in the sequence model is much higher than that of other types of neural networks.As the number of cycles increases,the bottleneck caused by the network structure of the recurrent neural network becomes more obvious.Aiming at the huge and complex calculation of the recurrent neural network in the sequence model,this paper uses hardware to parallelize the reasoning process of the recurrent neural network.This article uses programmable logic devices as an acceleration platform,and proposes multiple hardware parallelization improvement strategies for recurrent neural networks to improve the reasoning time of recurrent neural networks.Based on the hardware-based long and short-term memory network model,combined with the Neural Turing Machine mechanism,this paper proposes a parallelization scheme for software and hardware collaboration.Based on the above research work,this study uses the Neural Turing Machine mechanism to improve the accuracy of the sequence model on the public data set.On the basis of software improvement,the use of hardware to parallelize and accelerate the recurrent neural network has improved the inference speed of the neural network.This research has fully verified that the recurrent neural network combined with the Neural Turing Machine mechanism and the hardware parallelization acceleration scheme is an effective optimization scheme.