Research On Writing-in-air Handwritten String Recognition System

Texts play an important role in the human-computer interaction field, as one of the essential tools for information transmission and communication in human society. Currently, the acquisition modes of traditional on-line handwritten string like keyboards and touch screens have been unable to satisfy people’s requirements for the speediness and convenience of inputting text. Text input based on vision becomes a hot topic in recent years due to its advantages of intelligent input aroused concern. The visual text input mode based on vision can mainly be divided into the following two categories: one is that handwriting is produced by an ordinary pen on paper and a camera is used to capture the trajectory of the pen. Yet, due to this input mode needs the pen and paper, so to some extent, restrict users’ freedom. The other is that, the characters are written by a moving finger in the air. The finger trajectory is recorded by a camera or other motion sensors. Freedom of this kind of text input method uses higher realistic and thus has a more practical value. Since there is no pen lifting information in the moving trajectory, and the string constituted by one-stroke increases the difficulty of recognition. Therefore, string acquisition and recognition based on writing-in-air handwritten is a promising application prospect, but encounters huge challenge.This paper implements a handwritten string collection system writing in the air based on Kinect and puts forward a new method that written-in-air trajectory is recognized into string result. The major contributions of this dissertation are as follows:(1) This paper implements a handwritten string collection system writing in the air, sorting and calibration. Through gesture, users control when to begin and end the collection, the system tracks users’finger movement automatically. To support the research of written-in-the-air handwriting recognition, we have collected and annotated a database:including21people write7240numbers and14people write6700television channels (hybrid capital letters and numbers). An efficient annotation tool has been developed for processing the written-in-air handwriting data. The database can be used for typical research tasks of handwritten string analysis, such as handwritten string segmentation, handwritten character recognition, writer adaptation, and writer identification.(2)According to the recognition of the written-in-air handwriting string, we propose a smoothing algorithm and an over-segmentation algorithm. To solve the problem that there are extra stokes, due to the string by one-stroke, according to the geometric characteristics of the redundant segment, we propose redundant segment unary and binary geometric models, and propose a deletion category, which adds to characters classification.(3) We propose a new beam search algorithm based on lexicon-driven, in which we integrate character classification, redundant segment geometric model and character geometric models in the integrated segmentation-recognition framework to complete string recognition. Our method achieves promising results:the string-level correct rate achieves80.60%, which demonstrates the effectiveness of our method.