Study On Key Information Recognition Of Multi-type Forms Based On Visual Features

With the development of international trade and the advent of the information explosion era,forms have been widely used in the field of transportation logistics due to their simple and intuitive characteristics.At the same time,forms are becoming increasingly indispensable in financial and medical fields,such as logistics forms,receipts and resumes.With the intelligent and digital reform of all walks of life,it is urgent to realize the automatic extraction of multi-type forms,and to save the information in forms as structured data for easy retention and retrieval.Taking transportation logistics forms as an example,there are complex styles and various types of most international logistics forms in practical application.However,the key information extraction algorithms that have been put into application are usually only for forms in fixed formats or fields.There is still no reliable technical method to realize the key information recognition of multi-type complex forms.Therefore,this paper designs a key information recognition system for multi-type forms by analyzing the characteristics of multi-type logistics forms.The main work of this paper is as follows:(1)Since most forms exist in the format of scanned images,this paper studies image text detection and text recognition algorithms.Combined with the characteristics of text in form images,the network and process of different text detection and text recognition algorithms are analyzed and compared.(2)Based on the analysis of visual features of multi-type forms and the research of text detection algorithms,a multi-task learning network named Multi-TFC(Multi-task Network for Text Detection,Frame Extraction and Form Classification)is designed.The Network can complete three subtasks simultaneously,and the rapid addition of new categories can be achieved in a short time only by training the form classification branch of the network.Experiments show that the network can complete multiple subtasks more quickly while ensuring accuracy.(3)For forms of known categories,a key information extraction algorithm based on relative position templates is proposed,which can extract key information efficiently and accurately.For forms of unknown categories,a general key information extraction algorithm based on key information interrelated pair matching is proposed,which solves the problem of common algorithms that are not universal and expandable.(4)A multi-type form key information recognition system is established to complete the entire process from multi-type form images to structured key information.In this paper,a dataset of transportation logistics forms is established.Experiments show that the system has good performance and strong universality on the logistics form dataset,and is feasible in the task of extracting key information from multi-type forms.There are 54 pictures,10 tables and 64 references.