On discriminative semi-supervised incremental learning with a multi-view perspective for image concept modeling

This dissertation presents the development of a semi-supervised incremental learning framework with a multi-view perspective for image concept modeling. For reliable image concept characterization, having a large number of labeled images is crucial. However, the size of the training set is often limited due to the cost required for generating concept labels associated with objects in a large quantity of images. To address this issue, in this research we propose to incrementally incorporate unlabeled samples into a learning process to enhance concept models originally learned with a small number of labeled samples. To improve the convergence property of the proposed incremental learning framework, we further propose a multi-view learning approach that makes use of multiple features such as color, texture, etc., of images when including unlabeled samples. For robustness to mismatches between training and testing conditions, a discriminative learning algorithm, namely a kernelized maximal-figure-of-merit (kMFoM) learning approach is also developed.;A typical strategy for semi-supervised learning is to choose samples where an existing model is able to correctly predict their class labels with high confidence. However, these samples are not usually the best in terms of reducing modeling error as they are often too similar to already seen examples. In contrast, the proposed incremental learning framework selects unlabeled samples based on an expected error reduction function that measures contributions of the unlabeled samples based on their ability to increase the modeling accuracy. In the proposed framework, one of the essential components for robust estimation of the expected error reduction is a use of ensemble classifiers, such as a combination of a kMFoM classifiers and a spectral clustering based nearest neighbor (NN) classifier, etc. We demonstrate that, given an unlabeled example - when half of the classifiers in an ensemble predict the class label for the sample almost definitively, while the rest of them remain uncertain - the maximum value of the expected error reduction can be obtained. We generalize this result by developing iterative learning procedures that control the number of classifiers within the ensemble that should exhibit high confidence in their classification results when selecting unlabeled samples. We further improve the stability of the proposed framework by exploiting a class prior distribution so that a potential class imbalance problem can be reduced.;On the other hand, taking advantage of multiple features (e.g., color, texture, etc.) is vital to achieving a good image concept model. In a semi-supervised setting, a typical method to benefit from multiple features, known as multi-view learning, is to enforce concept models trained on individual features to generate the same prediction result. However, such enforcement is not always beneficial because different features might preferably indicate different class labels. Thus, in this dissertation, we propose a multi-view learning technique that exploits an agreement function, a function conveying our degree of belief that the individual models should agree upon their outputs. Then, we formulate a closed-form solution for a kernel function that represents an unified reproducing kernel Hilbert space (RKHS) with this agreement function.;Combining individual techniques, we also propose an integrated semi-supervised incremental learning framework, namely a discriminative semi-supervised incremental learning approach with a multi-view perspective, that takes advantage of both the power of multiple features and the expected error reduction function. In the integrated framework, multiple features extracted from images are combined through the kernel function. An ensemble of discriminative classifiers are then learned using the kernel function from which the expected error reduction function is computed. Based on the values of the expected error reduction, a set of unlabeled samples is chosen and exploited to enhance an existing model gradually. We conduct a set of experiments on various image concept modeling problems, such as handwritten digit recognition, object recognition, and image spam detection to highlight the effectiveness of the proposed framework.