Rotation Transformation Based And Stacked Sparse Autoencoder Based Ensembles Of One-class Extreme Learning Machines

Extreme learning machine(ELM)possesses merits of rapid learning speed and good generalization ability.It has been widely applied to the research of the classification problem.Due to the random initialization of connection wights,the network outputs of ELM are usually unstable.Similar to ELM,one-class ELM(OCELM)also has the disadvantage of output instability.Ensemble learning has been considered as an important research branch in the field of machine learning because it can enhance the stability and generalization performance of the model.In order to improve the output stability and generalization ability of OCELM,several component OCELMs are combined using ensemble strategies to construct the ensemble model of OCELMs.The main work of this dissertation is as follow.1.A method of selective ensemble of OCELMs based on rotation transformation is proposed.Two stages are required to construct the proposed method.First,several new training sample subsets are constructed by using the principal component analysis(PCA)based rotation transformation.These new training sample subsets are used to learn their corresponding component OCELMs,respectively.Then,the diversity measure based on the angle cosine is utilized to calculate the diversity of each OCELMs in the ensemble model.The component OCELMs with lower value of diversity are removed.For the testing samples,voting strategy is utilized to classify them as normal data or novel data.By comparing with the related methods on the benchmark data sets,the effectiveness of the proposed ensemble method is verified.2.A method of OCELM ensemble based on stacked sparse autoencoders is proposed.In order to obtain a sparser feature representation,a sparse autoencoder is constructed by adding regularization terms based on the Transformed-l₁ norm and the l₂₁ norm into the objective function of autoencoder.Basing on this,a stacked sparse autoencoder is constructed to improve the feature representing ability of sparse autoencoder.Moreover,to efficiently use the obtained feature information among all the bottleneck layers of the stacked sparse autoencoder,the features in each bottleneck layer are utilized to train their corresponding OCELM.The ensemble of OCELMs can thus be constructed.The experiments are conducted to compare the proposed ensemble method and its related approaches on the benchmark data sets.Experimental results show that the proposed ensemble method possesses better classification performance.