RandPG Artificial Neural Network And The Extended Researches

Traditional artificial neural network learning algorithm is limited for its time-consuming training.With the proposal of Extreme Learning Machine(ELM),the deadlock was broken down,that is,ELM can significantly reduce the training time of feed-forward neural network while guaranteeing considerable classification accuracy.However,like BP neural network,although they all have global approximation,once the training samples are relatively complex,Extreme Learning Machine needs more hidden neurons in order to ensure the learning ability,which inevitably leads to the complexity of the network structure.Random projection is one of the effective tool for processing massive and high-dimensional data.Therefore,in this thesis,based on the traditional artificial neural network,learning from the fast learning mechanism of Extreme Learning Machine,we design a more rapid artificial neural network.The thesis takes Plane-Gaussian network(PG)as the main research object,its hidden layer parameters are obtained by clustering algorithm,which leads to the longer training time and easy to get trap into local optimal solution.To solve its problems,this thesis proposes an improved algorithm named that Plane-Gaussian artificial neural networks based on random projection(RandPG)and further optimizes the network structure.The thesis mainly makes the following research results:(1)Learning from the learning mechanism of Extreme Learning Machine,we use random projection to determine the weight and bias of hidden layer activation function in RandPG,which avoid the clustering method in PG network and prove global approximation and classification ability in theory.Experimental results on classification and regression datasets show that: RandPG network not only follows the advantage of PG network,which is more suitable for classifying datasets sampling from plane-shaped distribution,but also significantly accelerates its learning speed.With the same small number of hidden neurons,RandPG network has higher training accuracy and testing accuracy than ELM for plane-shaped datasets,and they have comparable classification ability for some other datasets with unknown distribution.But on the complex network traffic datasets,the experimental results of RandPG show not very good performance.In terms of training time and testing time,the speed of RandPG and ELM is very fast.Moreover,during the experimental process,it is found that with random method,RandPG can jump out of the local optimum.(2)On the basis of the single-hidden-layer RandPG network,a novel Two-hidden-layer network strcture is designed,and RandPG network with two hidden layers is proposed(TRandPG).In TRandPG,the parameters for the first hidden layer are generated in the same way as RandPG,which are generated by random projection.As for the parameters for the second hidden layer,they are obtained by backward parameter calculation method.Compared with RandPG network,TRandPG achieves higher accuracy in classification and regression problems,and it indicates that with the increasing complexity of network structure,it is beneficial to improve the expression ability of neural network.(3)RandPG network is combined with Convolutional Neural Network and the CNN-RandPG model is proposed in the image recognition field to test the ability to solve practical problems.The specific approach is that Convolutional neural network is trained to extract image features,RandPG is trained to as a classifier.The experimental results on the MNIST datasets show that compared with the classical CNN network trained by BP algorithm,the proposed method can effectively improve the recognition rate under the same hidden neurons,and the whole recognition process only needs half the time of the convolution neural network.