Chinese Handwritten String Recognition Methods Fusion Based On Attention Mechanism

Handwritten text(string)recognition method is a technology that converts handwritten text into electronic text.In recent years,with the development of computer technology and the rise of deep learning methods,a variety of handwritten text recognition methods have emerged.They can be divided into two categories according to the idea of segmenting the input picture: string recognition method based on explicit segmentation(also called over-segmentation)and string recognition method based on implicit segmentation(also called segmentation-free),both methods have their own advantages and disadvantages.This paper intends to merge the two methods and take advantage of the advantages of the two recognition methods to learn from each other.After referring to the system fusion technology in other fields,this paper first proposed the neural network fusion model based on attention in the field of string recognition.The specific method is to send the recognition results of two Chinese handwritten text recognition methods to the same picture into our fusion model.The model extracts features through the convolutional neural network,and then combines multiple inputs through the attention mechanism to obtain the final fusion results.Different from the traditional fusion model method,this paper does not use the Recurrent Neural Network(RNN)structure,but uses a fully convolutional sequence to sequence(seq2seq)structure that can be calculated in parallel.In order to realize the many-to-one attention method in the case of full convolution,a variety of attention fusion realization methods are designed.This article also proposes a new fusion model training method,which is directly fused on the training set of the above two Chinese handwritten text recognition methods,so that no additional data will be used.This fusion model training method will make the experimental comparison more fair.The experimental part first analyzes the influence of different model structures on the fusion effect,and determines the optimal fusion model structure.It also proves that the fully convolutional fusion model using the joint self-attention mechanism proposed in this paper is superior to the traditional RNN structure fusion model.Then through comparative experiments on semantic information to explore the reasons for the improvement of the fusion method,it is found that the improvement of the fusion method depends on the semantic information in the training set,which shows that the main function of the fusion model is to correct the semantic errors in the recognition results.In addition,a fusion experiment in which the participating models are all string recognition models based on the implicit segmentation method is designed,which proves the effective complementarity of the two different string recognition methods.The experimental part proves the feasibility and effectiveness of the fusion of two string recognition methods based on explicit segmentation and implicit segmentation,but also exposes the defect of the fusion model's dependence on semantic information.At the end of the article,a variety of follow-up work improvement ideas were proposed for this defect.