Offline-online Skeleton Samples Co-training Based DBLSTM Handwritten English Recognition

Handwritten English recognition draws interests of researchers due to its practical potentials and research hardness. Recently the deep-learning and un-segmentation based methods give new life to this filed. Conventionally handwritten English recognition is divided into o?ine recognition and online recognition. Separate recognition systems are built to to deal with di?erent kind of data. In this thesis we will investigate the co-training methods of o?ine and online handwritten samples, by using the un-segmentation methods and deep learning frameworks.After normalizing the image contrast and correcting the slant, we build the recognition system based on DBLSTM-HMM hybrid framework. The basic concept is: firstly we train a GMM-HMM to do automatic-segmentation, then under force alignment algorithm we generate the mapping relationship between input frames and HMM states, which later are used to train a DBLSTM recurrent nerual network for classifying the new input frames. In the decoding phase, we combine the HMM transition probabilities, DBLSTM posteriors, lexicon contraints and language model to find the best decoding path, by using Viterbi algorithm. This system is verified to be useful on IAM o?ine dataset. We also optimize the network structure on it.To train the o?ine and online samples together, we render the online handwriting samples into static images, and use skeletonization methods to convert both o?ine and online samples into 1-piexel thick handwriting skeletons. This is to blur the boundary between o?ine and online samples. Then we built the DBLSTM-HMM system on merged o?ine-online skeleton handwriting samples. The merged samples can significantly enlarge the size of training samples, which can lead to a more robust system. From the experiments on IAM o?ine-online dataset and Microsoft INK online dataset, we can see recognition accuracy improvements on both o?ine and online test sets.Under DBLSTM-HMM recognition framework, this thesis innovatively uses skeletonization methods to merge o?ine and online handwriting samples. Experiments have proved the e?ectiveness of this method.