Research On Ensemble Of Random Weight Networks

Random weight network is a single hidden layer feedforward neural networks(SLFNNs),its input weights and hidden nodes biases are randomly generated,while its output weights are determined analytically.Extreme learning machine(ELM)network is a special random weight network and is a hot research topic in the field of machine learning recently.Extreme learning machine is a simple but efficient algorithm for training single hidden layer feedforward neural networks with fast speed and good generalization ability.ELM has been successfully applied to many fields,such as pattern recognition,computer vision,biological information processing,etc.However,there are two problems in ELM.The first one is architecture selection,the second one is prediction instability.(1)The method of ensemble of retrained extreme learning machine,which firstly several SLFNNs are trained by ELM,secondly the trained SLFNNs are integrated by majority voting method,finally the integrated model is used for data classification.(2)The method of ensemble dropout extreme learning machine,which is inspired by the dropout technique,but different from the usage of traditional dropout technique,this paper use dropout to determine the architecture of basic classifiers,and use ELM to train the basic classifiers,finally use fuzzy integral to integrate the trained basic classifiers.In order to verify the effectiveness of the proposed method,we experimentally compared the proposed approach with original ELM and ensemble ELM on multiple data sets,the experimental results and the statistical analysis of the experimental results show that the proposed approach outperforms ELM and EELM.Furthermore,the proposed method can address the first problem mentioned above and can improve prediction stability,and can address the second problem mentioned above to some extent.