Study On Finger-written Virtual Chinese Characters Recognition

As vision-based HCIs can develop'sense'for computers, they become a popular research field. If a computer with a camera can read our favorite means of communication, such as gesture, body language, handwriting, sketch, this would provide a natural, friendly and very effective vision-vased HCI.On the other hand, handwriting recognition modality has been widly intergrated in machines for the convieninece of human-machine interection. Handwriting usually is captured using a touch screen and a special pen. It is not convenient for mobile application when the size of screen is limited. Using camera to capture handwiring can break away from the limit of touch screen. Users can adopt the favorite means based a pen and paper when they communicate with a computer.The communication means based on a pen and paper is familiar for human being, but it is not convinient and environmental enough because it requires users bring a pen and needs lots of paper. If people can write characters virtually by just using the movement of his finger-tip on a common plane, and computer can recognized those characters, this will provide an interesting wireless character inputting modality for HCI application. As those finger-written characters can't be seen without ink information, we call them finger-written vitual Chinese characters (FWVCCs). This dissertation researches the reconstruction, coding and recognition of FWVCCs and presents a series methods.1.Background modeling is one important computer vision problem. To get the information of finger moving, we need to segement the'hand'from the video images.Considering the FWVCCs recognition system is a real-time system, we propose a simple but effective single-Gaussian background model to segement the'hand'. Experiments show the segmentation results based on this model are acceoptable.2.To reconstruct FWVCC real-time,this dissertation studies the application of the Flat Functional-link Neural Network (FFNN) to predict FWVCC moving trajectories. To solve the prediction problem of a non-stationary time series, conventional neural networks need a lot of time and samples to train, where FFNN can solve this problem very well. Considering the structure of Chinese characters, the dissertation makes some improvements for FFNN and chooses the appropriated train samples, and promising experimental results have been obtained. Furthermore a comparison is performed between the predictions of the Flat NN and a Kalman filter. Experiments suggest that the improved FFNN predictor works better for the prediction of trajectories of handwritten Chinese characters. 3.To get the trajectories of finger moving, this dissertation presents a fast and robust fingertip detection method. First, based on the analysis of the samples of hand contour, a candidate region for fingertip localization was selected. Then, the location of the fingertip was located based on circle feature matching and the candidate region. To demonstrate the strength of the method, the method was run on several sequences with varying light condition, different degrees of clutter background and different speeds of finger movement, experiment shows that the correct rate can reach 98.5%.4.We propose a FWVCCs reconstruction method based on Bezier curve coding . By this method, the strokes of FWVCCs appear smoother than connecting the trajectories of finger moving by lines. This method can also benefit to construct a small storage database but it can contain a lot of characters.5.At last, we also research the FWVCCs recognition, which includes preprocessing, feature extraction, classifiers design. Three new statistical classifiers are proposed, they are similar-Chinese-categories-based heirachy LDA classfier, Kernel Modified Quadratic Discriminant Function and MLDA+LDA classifier (modifier linear discriminate alalysis+LDA). The highest recognition rate of FWVCCs can reach 93%.In conlusion,the research of FWVCCs recognition is a multi-discipline,comprehensive research item, which can realize a new kind of more natural video based approach for inputting the handwritten Chinese characters, and is far-reaching significance in theory and application.