Research On Mongolian Online Handwriting Recognition Based On Fusion Model

The research on handwriting recognition always is a hot issue in the field of computer vision.With the increasing demand for digitization and information in people’s lives and work,handwriting recognition technology is becoming more and more valuable in practical applications.Nowadays the research on handwriting recognition for some languages with relatively clear word structure,such as Chinese,English and Japanese,is relatively mature.However,there is still relatively little research on languages with special textual structures such as Mongolian and Tibetan.The free writing and the huge vocabulary of Mongolian lead to poor and late research on its handwriting recognition.Therefore,we choose traditional Mongolian as the research object in this thesis,and use the deep neural network model as the architecture to realize the Mongolian online handwriting recognition.The specific research work of this thesis is as follows:(1)There are lots of common words in Mongolian,and it usually has special word structure,which makes it unsuitable to use a single word as the basic unit for online handwriting recognition in Mongolian.Therefore,we choose three methods of Mongolian word segmentation,which are named as twelve prefix code,presentation code and grapheme code.A multiscale model is proposed to evaluate three word segmentation methods for Mongolian online handwriting recognition.Through the comparative experiments,the grapheme code was selected as the segmentation method for Mongolian words.The problem of huge vocabulary recognition on Mongolian is effectively solved by using grapheme code.(2)For the feature of long input sequences of Mongolian handwritten words,a sequence-to-sequence model with an attention mechanism is adopted to realize Mongolian online handwriting recognition.The sequence-to-sequence model is good at dealing with the problem of inconsistent length in input and output sequences,and the introduced attention mechanism can effectively solve the problem that the model will forget some information due to the long input sequences.Experimental results show that the model achieves better performance on Mongolian online handwriting recognition compared with the multiscale model.(3)For the high out of vocabulary problem in Mongolian,a fusion model for Mongolian online handwriting recognition is proposed.A language model with a large vocabulary is pre-trained,and then the trained language model is fused into the sequence-to-sequence model with an attention mechanism.Three model fusion methods are proposed: former-fusion,latter-fusion,and whole-fusion.The experimental results show that all of three fusion models improve the accuracy of Mongolian online handwriting recognition,and the whole-fusion model has the best performance.Finally we study the problem of simultaneous training of the parameters of the language model,and the experimental results show that synchronizing the training of the language model parameters can further improve the accuracy of the fusion model for Mongolian online handwriting recognition.