Bayesian frameworks for deformable pattern classification and retrieval: Application to handwriting recognition

Model-based object recognition is a process in which an a priori model is searched for in an input image and subsequently its occurrence and location are determined. This top-down approach has long been proposed with the objective to extract a target shape from a noisy environment, which is achieved by integrating the segmentation and recognition processes via the underlying search process. Although there already exist some effective solutions for the problem, they only work under the assumption that the objects of interest are either rigid or subject to only global affine transformation. To deal with non-rigid objects, models capable of local deformation are required. In this thesis, we are interested in two related problems, namely shape-based pattern classification and shape-based pattern retrieval , using the deformable model paradigm in the context of handwriting recognition. The nature of the two problems is different in that the former tries to match multiple candidate models with the input data to find out its identity, while the latter tries to match one single input model (a query) with a set of candidate data patterns in order to rank them according to their shape similarity to the input.; Deformable models were recently proposed for handwriting recognition due to their ability to handle large shape variations. In this thesis, we first propose a Bayesian framework for deformable pattern classification as a unified approach for modeling, matching and classifying shapes. Different issues related to the system accuracy and efficiency for large-scale applications are studied. To evaluate the proposed framework empirically, we have performed extensive experiments using a benchmark dataset (from NIST) of real-world isolated handwritten digits. The accuracy is 94.7% using only a small set of deformable character models.; Research on the deformable shape-based pattern retrieval problem is still in its infancy. As the second part of this thesis, a Bayesian framework for deformable pattern detection is proposed with application to handwritten word retrieval. This new framework has the intrinsic property of finding a match with only part of the input (e.g., one character in a handwritten cursive word). This framework implements the integrated segmentation and recognition paradigm implicitly. By properly combining this framework with the previous one, a novel matching algorithm called bidirectional matching is proposed. Bidirectional matching outperforms the two individual frameworks in terms of matching robustness and accuracy, and bears a close analogy with Hausdorff matching. The algorithm has been applied to the CEDAR dataset for the retrieval of handwritten words. It achieved a correct matching rate of 85.5%, a recall rate of 59%, and a precision rate of 43%. Considering that the retrieval accuracy (in terms of precision and recall) of an existing system for the retrieval of even isolated images is only in the 30 to 60% and 15 to 20% range, respectively, we believe our results are encouraging and hence the proposed bidirectional matching approach is worth pursuing further research on.