Techniques For Local Feature Based Image Categorization

In this thesis, we consider the problem of local feature based image categorization. We analyze the components of a modern image classification system, and propose several methods for improving the discriminability of an image representation on two levels, namely, the feature level and the codebook level.At the feature level, we pay attention first to sampling strategy of local features. In the context of applications such as image categorization and image matching, the performance strongly depends on the location and the stability to illumination and geometric changes of local features. In view of this, we first propose a general approach for detecting affine invariant regions and develop a new affine detector based on the 4th differential invariant (DI4) under this framework. The new detector is first shown to be robust against geometric and illumination changes, and is further evaluated in comparison with various keypoint based detectors and dense sampling in image categorization task.Our primary focus at the feature level is to learn a linear discriminant projection (LDP) that can simultaneously reduce the dimensionality and improve the discriminability of a local image descriptor. We analyze in depth its properties under a broad framework of supervised linear dimensionality reduction, and provide insights into the relations between various state-of-the-art methods. A major limiting factor in the application of LDP is that it requires a large set of labeled training data. We propose a novel approach for simulating labeled train-ing data, and demonstrate the performance of such a scheme is comparable to that of using annotated training data. This makes it possible to apply LDP to problems where the ground truth of descriptor labels is not available, and as a result releases the true power of LDP. Such a simulation scheme is general in the sense that it can be applied to any supervised dimen-sionality reduction technique. We perform an extensive evaluation on standard datasets in the context of image matching and image categorization. We demonstrate that in all applica-tions considered, LDP with the proposed simulation scheme enables significant dimensionality reduction and at the same time boosts the performance.At the codebook level, our goal is to increase the discriminability of a codebook by means of machine learning. Considering that there is different discriminative ability of each codeword for image categorization, we propose a codebook weight learning approach to measure the discriminative ability of codewords. Inspired by recent advances in distance metric learning, we formulate the problem as one of learning a similarity metric. We show that such a problem can be cast as a constrained convex quadratic program (QP), and solve the QP efficiently with bootstrapping constraint selection and alternating optimization. Experiments on both synthetic and real datasets show that the proposed codebook learning method improves the classification accuracy, especially in the case where the codebook size is large compared to the number of training examples.We also investigate several other factors contributing to the performance of a codebook, including codebook generation technique, codebook size, assignment scheme, etc. Experi-ments show that the codebook generated with agglomerative clustering performs even worse than randomly sampled codebook in the case of hard assignment, while can outperform k-means and random sampling approach in the case of soft assignment. We also observe that using a larger codebook can improve the performance, however, increasing its size further beyond some point is not necessarily helpful. Due to more uncertainty introduced by more codewords, the soft assignment scheme can improve the performance of a large codebook more significantly.