Research On Extreme Learning Machine Algorithm Based On Adaptive Stochastic Resonance Mechanis

Extreme learning machine is a fast algorithm applied to the feedforward neural network,which has the advantages of fast learning speed,high fitting accuracy,and strong general approximation ability.Compared with the traditional gradient descent propagation algorithm,extreme learning machine can learn certain functions that the backpropagation algorithm cannot operate,such as the threshold function with zero gradients and differentiability.Threshold function is widely used in practical engineering problems because of its easy hardware implementation and low computational complexity.However,in the face of complex learning tasks,extreme learning machine algorithm still cannot optimize the threshold neural network well,and the network performance needs to be further improved urgently.In this thesis,the adaptive stochastic resonance mechanism and the extreme learning machine principle are combined to combine a hybrid algorithm for optimizing threshold neural networks,and endow the optimized network with significant importance for practical application.In this thesis,we first propose a hybrid approach for training threshold networks by combining the fast learning features of single-layer extreme learning machine and the noise-beneficial learning capability of adaptive stochastic resonance.The training phase randomly selects the input weights of the noise-modulated threshold network,and injects a set of noise samples into the threshold activation function,so that the noise-modulated threshold network has a non-zero gradient during the training phase,so that it can be updated using gradient descent including injected noise and network parameters.The injected noise finally converges to a non-zero positive value after iterations,and finally the output weight is determined as the least squares solution by extreme learning machine algorithm.This new hybrid approach not only finely optimizes weights in an extremely fast manner,but also adaptively searches for an optimal non-zero noise level.For function fitting and data classification,compared with the test classification accuracy obtained by a network composed of neurons using a continuous activation function(full precision),the threshold network can achieve the same or even higher classification accuracy with fewer hidden layer neurons.This also proves the benefit of noise in the threshold network: first,the addition of noise makes the neuron function of the threshold network transform into a smooth differentiable activation function,making it possible to be trained by the stochastic gradient descent algorithm;second,the appropriate noise intensity increases the test classification accuracy of the network.In addition,this thesis proposes a multi-layer extreme learning machine algorithm for training noise-enhanced autoencoders,using multi-layer stacked autoencoders for feature extraction,and injecting noise into the threshold activation function of each hidden layer during the extraction process.The threshold activation function can be smooth enough for parameter fine-tuning during backpropagation,and then the extracted features are used as new inputs to optimize and restore images through classic extreme learning machine.Experiments have confirmed that the excellent performance of the designed noise-enhanced autoencoder.The algorithms and findings presented in this thesis expand the prospect of adaptive stochastic resonance and extreme learning machine in practical machine learning tasks.