Form Location And Extraction Based On Graph Representation And Matching

Location and extraction of express waybill image is an important preprocessing in express sorting,only by understanding form information can we realize sorting of packages correctly.Currently existed form extraction methods are mostly according to simple layouts and fixed forms.They cannot adapt to images of different categories,brightness,scale,rotation direction,which are sensitive to interference information such as partial occlusion and complex background,all of above are difficult to achieve efficiently extraction.Therefore,a form location and extraction method based on graph representation and matching is proposed in this paper.Firstly,key regions such as existing printed patterns or character areas in reference form are chosen as nodes to build up the reference graph,which as an alignment in matching process.Secondly,candidate regions based on image segmentation in a test image are extracted,as the nodes of another graph,which is named test graph,and subgraph of test graph is called candidate isomorphic graph.Thirdly,the similarity between reference graph and candidate isomorphic graph is calculated according to graphs attributes.Finally,candidate isomorphic graph with maximum similarity is chosen as optimal matching of the reference one.Based on optimal matching,location mapping of reference form and test image is built up.Then,mapping transformation are performed for location in post-processing,text information of interest region in test form are extracted precisely.The main work of this paper includes:1)Propose form image graph presentation method,and turn form image matching problem into a graph matching.Test form images with unfixed content filling,complex parcel backgrounds and pattern.Global matching methods are easily affected by noise and foreign objects,it is difficult to achieve a good alignment performance.a flexible and robust graph representation method is adapt to overcome these problems.A form graph is constructed for representing form image characters,which nodes are derived local area block of form image.Image matching with graph representation is more effective for different categories,brightness,scale,direction,partial occlusion,interference background.2)Propose a method of build up a test form graph based on selective foreground segmentation algorithm.In order to matching key areas include form-specific character blocks,text lines,and patterns,a selective foreground segmentation algorithm is used in this paper.A Targeted foreground goals segmentation method is performed based on the selective search segmentation algorithm and a priori knowledge existed of the reference form graph nodes.Furthermore,complete the image matching with selecting a valid candidate node of test form graph.This algorithm is suitable for form images with similar gray scale texture features and layout rules,mixed foreground and background.3)Propose a graph matching algorithm based on structural characteristics.Traditional image matching algorithm is not suitable for matching images with cluttering background and foreground target texture approximation.After graph representation,this paper describes how to measure the similarity distance of the graph.Iteratively solves mapping functions and Reconstructs isomorphic graphs to improve the similarity of the two graphs by iteratively solves mapping functions,and then find the best match of the referenced graphs.The graph-based representation and matching algorithm proposed in this paper makes full use of the scattered local image blocks in the image to construct the form graph.By comparing the similarity of the graph attributes,form image matching is realized.Finally completes the location and extraction of the user's region of interest in the test form image.The experimental data set in this paper is derived from the image of the package form collected under the real courier sorter scenario.Experimental results indicate that the proposed algorithm has good performance on the express form images.Especially for the rotated,illuminated and partially shaded images,good robustness can be achieved.It is a general method and can overcome deficiencies of above methods.