| The Convolutional Neural Network(CNN)is a deep neural network that can automatically extract multi-level features of images and classify them.Because the CNN has the characteristics of weight sharing,sparse connection and pooling operation,it is widely used in target detection,face recognition and natural language processing.However,the current CNN algorithm generally has a high training time cost,which makes it occupy a large amount of computing resources,and at the same time it performs poorly on small sample data sets,and is prone to over-fitting.How to solve these two problems is also the research focus of the current CNN optimization algorithm.On the other hand,the Echo State Network(ESN)is a special recursive neural network(RNN).Its training process is efficient and simple,and it can effectively avoid over-fitting.It is widely used in timing signal processing and pattern recognition.However,due to the shallow and random nature of ESN,it has poor performance in dealing with complex features,which also limits the further application of ESN in engineering.The research in this paper finds that by combining CNN and ESN,the two will make their own advantages and make up for each other’s disadvantages.Therefore,this topic proposes two combination models from the perspective of optimizing the CNN model and optimizing the ESN model:1)The Echo State Network-Classification Based Convolutional Neural Network(E-CNN)model.The model replaces the fully connected layer in the CNN model with ESN and derives a new residual iteration formula.The Back Propagation algorithm(BP algorithm)is used to train the hidden layer parameters of the CNN part.Based on the linear regression rule,the output weight of the ESN is trained.The feasibility of the new algorithm is proved by the simulation experiments on the MNIST dataset,the ORL dateset and the Fashion MNIST dataset.The experimental results show that the model not only retains the ability of CNN multi-level feature extraction,but also reduces the required training time of the algorithm and improves performance on small sample data sets by introducing the ESN module.2)The Convolutional Neural Networks-Feature Extraction Based Echo State Network(C-ESN)model.The C-ESN model introduces the convolutional layer and the pooled layer in the CNN network into the ESN model as its feature extraction part.To preserve the advantages of ESN,the model uses the pre-trained CNN model and continues to follow the linear regression rule to train the output weight of the ESN.The feasibility of the new algorithm is verified by simulation experiments on the datasets MNIST and Fashion MNIST.At the same time,the experimental results reflect that the new model not only retains the simple and efficient ESN training,but also compensates for the shallowness and randomness of ESN by introducing CNN module. |
