Sparse representations for image classification

In recent years, filter bank based approaches such as wavelet and wavelet packet transforms have been used extensively in image classification. One key issue in wavelet based classification methods is how to choose the best set of features (subbands). In the existing methods, each subband is evaluated separately or only the children subbands are compared with their parent subband for selection. In this thesis, we show that this subband selection method does not consider the dependence from different subbands in the selection process. In order to address this issue, we propose subband selection methods that take the dependence into account. We offer a theoretical and experimental analysis of dependence among features among different subbands. Based on this analysis, mutual information based subband selection (MISS) algorithm is proposed for subband selection based on feature dependence. The MISS algorithm is further improved by the subband grouping and selection (SGS) algorithm which combines the dependence between subbands and the evaluation score of each subband. All of these methods result in a compact set of features for efficient image classification.; The development of efficient subband selection methods for image classification motivates us to consider the more general problem of sparse representation of images for classification. We propose a new approach in the framework of the sparse representations by combining the reconstruction error, classification power and sparseness in a single cost function.; The formulation of the proposed sparse representation for image classification method is further improved by using the large margin method for the measure of discrimination. Based on this new and improved formulation, we can model the robust and sparse feature extraction with an optimization problem that can be solved by iterative quadratic programming. In order to reduce the computational complexity required for the iterative quadratic programming, we propose decomposing the robust and sparse feature extraction into two steps, with the first step being sparse reconstruction and the second step being sparse feature selection and dimension reduction. For the second step, we propose a new method called large margin dimension reduction (LMDR). LMDR integrates the idea of L1-norm support vector machine (SVM) and distance metric learning for obtaining feature representation in a low dimensional space.; Finally, to measure the goodness of features obtained by different methods, a mutual information based feature evaluation criterion is proposed. The proposed measure is independent of distance metric and classifier. Computation of mutual information in a high dimensional space is addressed by using the uncorrelated linear discrimination analysis (ULDA). The proposed computational model effectively reduces the computational complexity of computing mutual information for feature evaluation.