A Study Of Dimensionality Reduction And Its Application On Face Detection

Dimensionality Reduction is a vital research direction in the field of machine learning, it directly determines the quality of sample representation, which is one of the most important part in the learning framework. A good representation of samples helps improve the computation efficiency of models, reduce the consumption of storage, improve the performance of learning algorithms, reduce the sensitivity of algorithms against parameters as well as enhance the interpretability of data. Moreover, with the advent of the era of big data, when large amount of data are accumulated and the dimensionality of features rockets, the importance of dimensionality reduction algorithms is also increasing dramatically. So as the requirement on these algorithms. Two of the most acute problems are: Firstly, with the expansion of data scale, the cost of sample labeling rises, thus leading a urgent demand on unsupervised dimensionality reduction algorithms; Secondly,the rise of data quantity sharply increases the computational and storing burden, which strongly threatens the applicability of the traditional dimensionality reduction algorithms.In order to solve the above problems, in this paper we propose an unsupervised feature selection framework as well as an efficient feature transformation framework. Furthermore,we also apply dimensionality reduction into face detection and achieves an application of dimensionality reduction algorithm. Concretely, the work in the paper can be roughly divided into three parts:(1)We propose a spectral clustering-based global and local structure preservation unsupervised feature selection framework Based on the idea of conserving both global similarity information and local geometrical structure simultaneously, we propose an unsupervised feature selection framework in the second section. The global information of data is preserved by introducing spectral clustering, who can automatically extract the classing information among samples. As to local geometrical information, we conserve it by manifold learning in this paper. To solve the resulting optimization problem,we propose a convergent iterative optimization algorithm. As the local information in the framework can be conserved by different manifold algorithms according to the specific demand of different data sets, the proposed framework is flexible and with a strong adaptability against different sample sets. Comprehensive experiments prove the effectiveness of two instantiation algorithms of the framework.(2)We propose a selection-based random fourier supervised feature transformation algorithm In the third section, we propose a two-step supervised random fourier dimensionality reduction algorithm. Different from the traditional feature transformation algorithms who generate the essential dimensions in data-driven manners, the proposed algorithm generate the dimensions randomly and then searching the the optimal random feature subsets by feature selection algorithms. Specially, random features are firstly pruned by a novel ?2,1-Norm Extreme Learning Machine in the first step and then an optimal random feature subset with largest relevance and least redundancy is selected by Joint Mutual Information algorithm. It is worth noting that our work bridges the gap between dimensionality reduction and feature selection, which means all the feature selection algorithms either supervised or unsupervised algorithms can be used to fulfil the task of feature transformation. Experiments on multiple benchmark datasets demonstrate the superiority of the proposed algorithm in both terms of classification accuracy and computational efficiency.(3)We propose an improved cascaded face detection algorithm In this part of the paper, we propose a novel texture feature called Multi-Block Local Gradient Patterns. The newly introduced feature is an expansion of Local Gradient Patterns. Comparing with the original version, the new one not only inherits the good quality of the previous version but also gains stronger robustness against noise as well as better structure description capacity. Based on the Multi-Block Local Gradient Patterns, we integrate Ada Boost algorithm with Extreme Learning Machine to construct the cascaded classifier for face detection. The added ELM can be trained in a fast manner, thus reduce the training time consumption of Ada Boost algorithm in the last cascade. Experiments on CMU+MIT data set demonstrate the performance of the proposed algorithm.