Flexible statistical modeling

The support vector machine is known for its good performance in two-class classification. The topic of this thesis is based on two variations of the standard 2-norm support vector machine.; In the first part of the thesis, we replace the hinge loss of the support vector machine with the negative binomial log-likelihood and consider the kernel logistic regression model. We show that kernel logistic regression performs as well as the support vector machine in two-class classification. Further more, kernel logistic regression provides an estimate of the underlying probability. Based on the kernel logistic regression model, we propose a new approach for classification, called the import vector machine. Similar to the support points of the support vector machine, the import vector machine model uses only a fraction of the training data to index kernel basis functions, typically a much smaller fraction than the support vector machine. This gives the import vector machine a computational advantage over the support vector machine, especially when the size of the training data set is large.; In the second part, we replace the L2-norm penalty term of the support vector machine with the L 1-norm penalty, and consider the 1-norm support vector machine. We argue that the 1-norm support vector machine may have some advantage over the standard 2 norm support vector machine, especially when there are redundant noise features. We also propose an efficient algorithm that computes the whole solution path of the 1-norm support vector machine, hence facilitates adaptive selection of the tuning parameter for the 1-norm support vector machine.