The Research Of Embedded Stacked Sparse Autoencoder Feature Fusion Ensemble Algorithm

Learning high-quality features with strong discriminative ability is one of the key issues in classification tasks,which is crucial to the training and performance of a classification model.Traditional feature learning algorithms mainly including feature selection and feature extraction,both of which belong to shallow learning.Compared with traditional algorithms,deep neural networks have better capability of nonlinear fitting and feature extraction.Among them,stacked autoencoder(SAE)is a kind of representative deep learning method,which not only has the ability of feature selflearning,can mine the key information in the data through multiple layer nonlinear transformation,but also has the advantages of simple construction process,easy to understand and realize,etc.Therefore,it is widely used in the feature learning stage of classification.Although the current researches of SAE have achieved certain success,further improvement is still a challenging problem.The training of the SAE network relies on a large sample size.Therefore,with a small sample size,the network is prone to overfitting,resulting in limited representation capability of the extracted features.The original features and deep features describe the sample from different angles,so the fusion of them is expected to improve feature quality.However,the existing SAE network does not consider the complementarity between deep features and original features in the modeling process,and the fusion strategy still needs to be optimized,thus limiting the effectiveness of feature fusion.In addition,existing autoencoders have not considered the spatial information of samples in the feature learning process.Aiming at the above problems,this thesis intends to study the embedded stacked group sparse autoencoder feature fusion ensemble algorithm,which introduces the original features into the training process of SAE to enhance the complementarity between the deep features and original features,thereby improving the performance of feature fusion.In addition,on the basis of the autoencoder improvement and feature fusion,this thesis further considers the structural information of the sample space,and studies the embedded stacked group sparse autoencoder ensemble algorithm based on sample clustering,and realizes sample and feature collaborative learning.The main work of this thesis is as follows:(1)Aiming at the problem that the existing SAEs are prone to overfitting for small sample amount and the quality of the learned features is not satisfactory,an embedded stacked group sparse autoencoder feature fusion algorithm has been proposed in this thesis.First,the SAE network structure has been improved,an embedding unit has been added between two adjacent hidden layers,so that the original feature information has been introduced into the training process of the autoencoder network.Secondly,the deep features learned by the improved aotoencoder are combined with the original features,and then the L1 regularized feature selection is used to obtain the optimal hybrid feature subset.Finally,the hybrid feature set is randomly sampled,and weighted local discriminant preservation projection(w?LPPD)is used in each subspace,and then the base classifiers are trained and ensemble.The algorithm makes the original features participate in the training of the network by the embedding unit,which enhances the complementarity of the deep features and original features,and obtains high-quality features by effective fusion strategies,thereby improving the classification accuracy,stability and generalization of the model.(2)Aiming at the problem that the existing SAEs do not consider sample spatial structure information in feature learning,this thesis proposes an embedded stacked group sparse autoencoder ensemble algorithm based on sample clustering.First,iterative mean clustering is used to construct multi-level sample spaces to mine the structural information of the sample space.Then,the embedded stacked group sparse autoencoder feature fusion model is trained with the new samples generated by clustering in each sample space.Finally,the classification results obtained in each space are ensemble by decision fusion.This algorithm considers the structural information of the sample space in the feature learning process of SAEs,and performs sample learning and feature learning at the same time to realize sample and feature collaborative learning.Experimental results show that ensemble multi-level clustering space can effectively improve the classification accuracy.