Research On Text Localization And Recognition Of Bills Based On Deep Learning

As material and spiritual life becomes more and more abundant,people use various types of bills including shopping,dining and transportation in many life scenes.However,in the process of bills information review and financial reimbursement,dealing with massive bill information is a tedious and repetitive task for the finance staff.In recent years,text localization and recognition based on deep learning has become an hot research topic.Using text localization and recognition technology to accurately and quickly locate and identify the texts on the bills is of great significance for liberating manpower and improving the efficiency of the company and related personnel.Most of the scanned document images have poor text features,different fonts,different sizes,and dense lines,accompanied by noise interference in areas with seals and similar characters.Because of printing quality and other reasons,blurred texts will appear on some bills.In text localization,since the texts in the bills have a large difference in characteristics from the objects targeted by general object detection,the general object detection methods are easy to locate the upper and lower adjacent rows of the target text line in the text-intensive area of the bills.In addition,if the text localization result box does not contain the text tightly enough,the location of non-text areas will bring difficulties to subsequent recognition.In text recognition,the difference between the training data and the ticket characteristics will cause the model to get a lower recognition rate,data enhancement is required in conjunction with bill text features.Regarding the issues above,the main work of this paper is:(1)In order to overcome the shortcomings of overlapping and inaccurate localization results in text-intensive areas,the DTLN(Dense Text Localization Network)framework for dense text localization is proposed.It mainly includes three innovations.The first is to consider the IoU(Intersection over Union)localization confidence in network training,combined with the general classification loss and regression loss to determine the location of the prediction result boxes.Then,in the sample selection stage before network training,the CMax-OMin(Maximize Corresponding samples and Minimize the Others)strategy is proposed to optimize the quality of the samples to avoid overlapping situation of text results.Finally,the outputs of the neural network are finetuned by training a simple regressor to make them closer to the real text areas.Experiments prove that the proposed DTLN framework is superior to other methods in the texts of collected bill dataset and the general scene text dataset,especially on the bill dataset.When the IoU threshold is chosen to be 0.5 and 0.7,the F1-measure value is reached 0.87 and 0.65.We further performed ablation experiments to demonstrate the effectiveness of the proposed strategy.(2)An end-to-end bill text recognition model based on DenseNet,BLSTM(Bidirectional Long Short-Term Memory)and CTC(Connectionist Temporal Classification)is designed and implemented.The unique dense connection mechanism of DenseNet network plays the role of feature multiplexing and transmission.BLSTM can be combined with text context information for auxiliary identification.Finally,the output prediction result of the whole sequence is directly obtained through the CTC loss function.In order to incorporate the learning of the characteristics of bill texts into the recognition model,this paper uses the common texts of the bills to construct the bills corpus,and adds perspective,fuzzy and other transformations to the different background textures,fonts,colors and other features presented on the bill images,to design and generate a bill text recognition dataset that simulates the bill image scene.Experiments show that the DenseNet network used is superior to the general convolutional neural network in the feature extraction of bill text.The model integrated with BLSTM and the model trained using synthetic dataset can further enhance the recognition effect.(3)An easy-to-use platform for bill text localization and recognition system is built.The system incorporates the localization and recognition models proposed in this paper,as well as preprocessing techniques such as image contrast enhancement and stamp removal.