| Computer systems that can recognize patterns from data have become more and more important due to the large demand for such systems and the availability of massive computing power. This dissertation investigates a variety of topics in the area of pattern recognition. The main contributions include several state-of-the-art learning algorithms for pattern classification, a probabilistic model for cursive handwriting recognition, and a real-time computer vision system for vehicle tracking. In this dissertation, we formulate the pattern recognition problem in a statistical framework and focus on the design of models that can generalize to unseen data from a finite number of observations. In the statistical framework, the fundamental challenge of the problem is captured by the so-called approximation error/estimation error trade-off, or equivalently, the bias/variance dilemma, depending on which performance metric one is interested in. Given the particular problems we are trying to solve and the information available, we develop different methods to improve the recognition performance by overcoming the limitations imposed by the approximation error/estimation error trade-off and the bias/variance dilemma. In the case of non-parametric pattern classification, where all the information available is the training data, we develop a locally adaptive nearest neighbor classification rule and a minimum sphere covering approach to pattern classification. The locally adaptive nearest neighbor rule demonstrates better generalization performance than the original nearest neighbor rule because it adapts the metric locally so that nearest neighbors identified according to the new metric are more likely to have the same class label as the query point. The minimum sphere covering approach to pattern recognition provides another way to build RBF and logistic function classifiers by explicitly balancing the trade-off between complexity and empirical accuracy of the model. Experimental results on benchmark datasets demonstrate that both methods achieve state-of-the-art performance. In the case of cursive handwriting recognition, we develop a probabilistic model that combines our prior knowledge about the spatial arrangement of letters with outputs from local letter detectors to improve cursive handwriting recognition. In vehicle tracking, our method combines the Kalman filter and spatial context to achieve robust real-time tracking of vehicles. |
