Studies On Cascaded Deep Learning Model Based On Feature Extraction And Feature Selection

Feature learning is an important branch in machine learning, via which ?good? features can be extracted to provide semantic and structural information of data, leading to better results even for simple classification models. Instead of extracting features using raw observations or hand-craft methods, deep learning aims to extract features automatically, which can commonly provide high-level semantic and structural information. Moreover, with the help of multi-layer structure, deep learning models can abstract the data from different scales, meeting the needs for kinds of machine learning tasks. Currently, all the deep learning methods grow up based on the multi-layer scheme, which are roughly composed of three steps, i.e., feature transformation, non-linear operation and feature selection. However, most of the existing deep learning methods only concentrate on the two dimensional data with spatial structural information, leaving the general vector-form features unresolved.In this work, we propose a generalized cascade deep learning model to solve the feature learning problem with the general vector-form. To achieve this, firstly we propose a general unsupervised feature selection model, where L2, p norm is introduced to investigate the self-representation property of feature space more deeply together with its non-convex solutions, thus leading to an effective feature selection model.By combining generalized feature transformation methods with the proposed feature selection method, a cascade deep learning model for feature learning is finally obtained in our work. To fully utilize the features learned from each layer of the deep learning model, we propose a reasonable feature combination strategy, which adequately exploits the complementary information from different layers, thus improving the classification performance significantly.As to the training of the proposed cascade deep model, we propose a data augmentation strategy on the handwritten digitals dataset for better performance. During training the deep model, we study the performance of the proposed model with different data augmentation parameters. Experimental results have shown that the proposed data augmentation strategy boosts the performance notably.