Design And Implementation Of Single Node Hardware Reservoir Computing

Compared with traditional computer,artificial neural network has obvious advantages in solving the problems of time series prediction,image recognition and so on.Artificial neural networks can be divided into feedforward neural network and recursive neural network according to their topological structure.The feedforward neural network is suited for static tasks and has a narrow range of applications.The recursive neural network,by adding feedback signals,makes the network capable of remembering past history and has a wider range of uses.However,the recursive neural network involves more parameters in the training process,which leads to the complexity of the training algorithm,the training time is longer,and not easy to converge.Based on the recursive neural network,the researchers further proposed a reservoir computing,which consists of an input layer,a reservoir layer,and an output layer.The hidden layer in the recursive neural network is replaced by a thinly connected network structure known as the reservoir layer.The connection weights between the input layer and the reservoir layer and the connection weights between the nonlinear nodes in the reservoir layer are pre-generated and fixed.The only parameter needed to be trained is the output weight between the reservoir layer and the output layer,which radically reduces the number of parameters needed to be trained and shortens the training time.It overcomes the problem of the traditional artificial neural network training.With the further development of reservoir computing,the single-node reservoir with delay feedback was proposed.It can realize the high-dimensional conversion of input signals in time domain by time division multiplexing technology,and reduce the number of non-linear nodes to one,effectively reducing the resource consumption.The research lays a foundation for large-scale parallelization,integration and miniaturization of the reservoir.A single-node reservoir structure with delay feedback is proposed based on the theory of autonomous Boolean networks,and the corresponding mathematical model is established.The nonlinear node of the reservoir is only composed of two logic gates,which greatly reduces the difficulty of physical realization and the consumption of hardware resources.According to the different types of logic gates,the proposed reservoir is divided into four categories,and a comparative study is carried out.The characteristics of fading memory and separation of the four types of reservoirs are studied by applying different external excitation to the reservoir layers.The MNIST handwritten digital image recognition task is completed by using the reservoir computing system.Firstly,the system input layer processes the binary image.Then,the reservoir layer completes the nonlinear mapping to the image data by virtue of the nonlinear characteristic of the logic device itself.Finally,according to the state of the reservoir,the output weight is trained by ridge regression algorithm,and the winner-takes-all strategy is used to classify the output and realize the recognition of handwritten digital image.According to the established mathematical model,the influence of the four kinds of reservoirs’ hyperparameters on the recognition accuracy is compared,including the feedback delay time,the reset cycle and the number of virtual nodes.The simulation results show that the correct recognition rate of the four types of reservoirs is higher than that of the linear classifier(86%)by selecting appropriate hyperparameters.The single-node reservoir realized by the nonlinear nodes composed of an XOR gate and an OR gate achieves a maximum recognition rate of 91.30%.The proposed reservoir structure is implemented on the FPGA platform,and the data preprocessing of the input layer and the training and classification of the output layer are completed on the PC.Under the guidance of simulation experiments,the hyperparameters of the four kinds of reservoirs are optimized,including the feedback delay time and the number of virtual nodes,and compared with the correct recognition rate of the linear classifier.The experimental results show that four types of single-node reservoirs with delayed feedback only need one virtual reservoir state under appropriate feedback delay parameters.It can get a better recognition accuracy than the linear classifier.The maximum recognition accuracy of 91.72%can be obtained by increasing the number of virtual nodes.This is comparable to the accuracy of the multi-node reservoir computing with fewer logic devices.In addition,the effect of the number of training samples on the recognition accuracy is also studied.When the number of virtual nodes is 1 ～ 3 and the number of training samples is 10000(only 1/6 of the number of training samples needed for linear classifier),the correct recognition rate of the four types of reservoirs is higher than that of the linear classifier.