A Study Of Network Structure Optimization Method For Multi-layer Extreme Learning Machine Based On Particle Swarm Optimization

With the development of artificial intelligence technology,deep neural networks have received widespread attention.As a special deep neural network,the multilayer extreme learning machine has attracted attention for its fast calculation speed and good feature extraction ability.Hierarchical Extreme Learning Machine(H-ELM)is an emerging learning algorithm for generalized multi-hidden layer feedforward neural networks.Its training architecture is structurally divided into two separate stages: unsupervised hierarchical feature representation and supervised feature classification.However,due to its manually designed network structure,researchers need to spend a lot of effort to adjust the structure,which is an error-prone and time-consuming process.Particle swarm optimization(PSO)algorithm is famous for its fast search speed and high operation efficiency,and it is based on the concept of "population",making it easy to store a variety of network structures.Therefore,a H-ELM structure search method based on particle swarm optimization algorithm is proposed to search for H-ELM with high convergence accuracy and compact structure.Based on the previous work,Design and implementation of a H-ELM system based on PSO.The main work of this paper is as follows:(1)An H-ELM structure search algorithm based on multi-swarm PSO(MSPSO-HELM)is proposed for searching for H-ELM with high convergence accuracy and relatively compact structure.The algorithm is based on a standard PSO,with the goal of minimizing the error rate,supplemented by the principle of structural minimization,and searching for the optimal network structure.The network structure of the algorithm is represented by multiple swarm methods.Because the network structures of different hidden layers are stored in different sub-swarms,and they are independent from each other during the optimization process,they cannot exchange information with each other.Therefore,an improved particle swarm optimization algorithm is proposed so that neural networks with different hidden layers can learn from each other.The method is divided into intra-group learning and inter-group learning.Intra-group learning is directly optimized by standard PSO algorithms;inter-group learning is optimized by first aligning the vectors by filling and truncating operations.At the same time,the principle of minimizing the structure is proposed for ensuring the compactness of the network,and the network structure is further optimized to make it more compact.Finally,Experiments onmultiple single-class and multi-class data sets show that,compared with the optimal network structure in the H-ELM structure randomly generated 50 times,the network structure obtained by the method has higher convergence accuracy and relatively compact structure.(2)An H-ELM structure search algorithm based on multi-objective PSO(MOPSO-HELM)is proposed for searching for H-ELM with high convergence accuracy and relatively compact structure.The algorithm is based on a multi-objective particle swarm optimization algorithm,and simultaneously aims to minimize the error rate and the total number of network nodes to search for the optimal network structure.The network is represented by a two-dimensional matrix.According to the meaningless empty structure(Z-structure)generated in the process of optimizing the neural network,a Z-structure regeneration method is proposed.This method improves the ability of the particle swarm optimization algorithm to jump out of the local optimum while ensuring that the structure is meaningful.Because there are some solutions with too high error rates and too few nodes in the archive,which affect the search direction of the algorithm.A secondary optimization method for archives is proposed to locate solutions with too high error rates and delete them,so that the algorithm can be applied to the structure is optimized with higher precision and compactness,which improves the convergence speed of the algorithm.Experiments on multiple single-class and multi-class data sets show that this method can obtain a neural network with high classification accuracy and a simpler structure than MSPSO-HELM.(3)An H-ELM system based on PSO is designed and implemented.The system integrates the two algorithms mentioned above.By designing the algorithm operation module and the network design module,the algorithm operation,storage,and structure optimization and analysis are implemented.The feasibility of the algorithm is verified through actual tests.