Study Of Multiview Feature Selection And Dimension Reduction Techniques With Applications

Most previous works for multiview data analysis only focused on the algorithms of multiview classification and clustering, but rarely involved the problem of multi-view feature selection and dimension reduction. Recently, the rapid development of data acquisition equipment causes the dimensionality of multiview data to increase dramatically. Thus, building a classifier (or a clustering model) directly on original high-dimensional data, which might leads very high computational burden or poor per-formance, will usually not satisfy the requirement of particular tasks in real cases. S-ince features of different views are often correlated, to select significant features (or to learn low-dimensional representation shared by different views) is helpful for not only improving the accuracy and efficiency of the subsequent learning models, but also un-derstanding the complex cross-view relations. Therefore, the thesis follows the study of multiview feature selection and dimension reduction technologies with applications. The details can be summarized as the following four folds:(1) Joint local-and-global sparse multiview feature selection:to solve the prob-lem of selecting the discriminative features among different views for better perfor-mance (previous studies only selected class-related features), we introduce a novel selection metric for discriminative features and thus propose a novel Multiview feature selection method via joint local PAttern-Discrimination And global Label-relevance analysis (namely mPadal), which systematically includes local selection phase and global selection phase. Specifically, in the local selection phase, the discriminative features will be first selected by taking into consideration the local neighbor structure of the most discriminative instances, while in the global selection phase, the features with the topmost label relevance, which can well separate different classes in the cur-rent view, are selected. The global selection phase can be mathematically modeled and solved by employing the constrained quadratic optimization methods. Experimental results show that compared with several baseline methods in publicly available activity recognition data set IXMAS, mPadal performs the best in terms of the highest accura-cy, precision, recall and Fl score. Moreover, the features selected by mPadal are highly complementary among views for better classification results.(2) Rank minimization-based multiview feature selection:to solve the problem of measuring sample significance (previous studies simply assign the equal weight for a certain view, but different samples in multiview often have different discrimination ability), we measure the sample significance by introducing instance-specific weights. Here, a novel feature selection algorithm, Multiview Rank Minimization-based Lasso (namely MRM-Lasso) is proposed, which jointly utilizes rank minimization and sparse learning for measuring the significance of view-level, sample-level and feature-level. In particular, we first extend Lasso to sparse multiview feature selection task. By using the instance-specific weights, the latent correlation among different views can be suc-cessfully captured with low-rank assumption. Also, a feasible alternating optimization strategy via the Alternating Direction Method of Multipliers (ADMM) is employed for MRM-Lasso optimization. Experiments on IXMAS data and three benchmark data sets show that features selected by MRM-Lasso have better multiview classification performance than the existing baselines.(3) Incomplete data-oriented multiview dimension reduction:most previous s-tudies failed in the case that several samples in partial views are missing. Thus, we propose a novel multiview data recovery method, Sparse low-Rank Representation on multiview common Subspace (namely SRRS). Specifically, SRRS measures the intra-view relations by sparse and low-rank learning, as well as inter-view relations by mul-tiview common subspace learning, with the goal of recovering the missing samples with the given observed samples. Also, by using the proposed SRRS, three specific multiview dimension reduction methods, Multiview subspace learning via Graph Em-bedding (namely SRRS-MGE), Multiview clustering via Structured Sparsity (namely SRRS-MSS), and Multiview feature selection via Rank Minimization (namely SRRS-MRM), are presented for dealing with incomplete multiview data. Experiments on incomplete multiview data (i.e., multi-angle video activity analysis and multi-modality neuroimaging data diagnosis) show the promising performance of our methods, com- pared with the state-of-the-art methods. Also, the results of simultaneously integrating the sparse and low-rank assumption are better than that of using each of them separate-ly.(4) Online multiview subspace learning:most previous studies for multiview sub-space learning usually failed in the case of analyzing time series multiview data. To this end, we present online learning methodology, and first attempt to propose an Online Multiview subspacE Learning method via group structure (OMEL). It is commonly observed that the group structure of multiview data usually varies with the time chang-ing. Thus, in particular, group information of multiview data is measured by using both group sparsity and group constraints, with aiming to preserve the original group struc-ture in the learned subspace. For newcoming samples, subspace learning model will be updated to prevent that the previous data will be processed for many times. Typically, for effectively optimize the proposed OMEL, we employ the alternating optimization strategy, and also adopt the Greedy Coordinate Descent for solving the common low-dimensional representation across different views. Finally, we extensively evaluate the proposed OMEL on visual object tracking with 15 benchmark video sequences, by comparing with seven state-of-the-arts algorithms. Experiments show that OMEL is robust and excellent to deal with challenging tracking problems (i.e., shape defor-mation, background clutters, object occlusion and illumination variance). Moreover, several evaluation tests are conducted to validate the efficiency of group structure as-sumption.