| In recent years,artificial intelligence,big data technology and computer hardware integration and storage technology have achieved rapid development.This has undoubtedly triggered a new generation of industrial revolution and spawned many emerging industrial technologies represented by driverless driving,smart cities,virtual reality,human-computer interaction and smart medical treatment.In addition,industries,especially the explosive growth of data in the medical field,put forward new requirements for digital storage and analysis.In the era of "Internet + healthcare",deep learning can realize intelligent processing of massive laboratory data with the help of big data.Among them,as the presentation form of medical examination report,the laboratory test sheet contains a variety of physical examination information of patients,which plays an important role in disease analysis.Deep learning is used to make the machine recognize the text information of the laboratory list,and then big data is used for disease analysis,so as to realize the intelligent medical auxiliary diagnosis system.Table recognition is the first step for machines to understand medical examination reports,and it is also an urgent practical problem to be solved.The main task is to analyze the table structure of the medical examination reports,identify the content of the cells,and reconstruct the images in natural scenes into a spreadsheet for further storage and analytical processing.Because the Chinese medical examination reports is a wireless form with complex structure,and contains various types of characters,it is still a difficult problem to realize its accurate recognition.This thesis proposed a table recognition network(CoT＿SRN)for medical examination reports based on Contextual Transformer.The proposed algorithm consists of two main parts:CoT＿encoder and SRN＿decoder.The CoT＿encoder is combined with the Transformer module that can make full use of the context information.And the CoT＿Net50 backbone network that is more suitable for wireless complex Chinese medical examination reports table recognition is constructed to extract and encode the input image features.Through the Attention+GRU structure,the SRN decoder forms the attention-head of sequence recognition,predicts the encoded feature sequence,and outputs the table structure sequence and cell position information prediction.Finally,the test sheet image is reconstructed into a complete spreadsheet by aggregation with the text detection and text recognition results.This thesis also proposed a key information extraction algorithm for Spatial Dual-Modality Graph Reasoning based on STC＿Unet to realize medical examination reports table recognition.The algorithm consists of three parts: Dual-Modality fusion module,Graph Reasoning module and classification and post-processing module.The visual features were extracted through the STC＿Unet network combining the Swin＿Transformer and Unet.The semantic features were extracted through the Bi＿LSTM.And then the visual and semantic bimodal features were fused through the Kronecker product.The above features are input into the spatial reasoning model to extract the final node features,and finally the multi-classification task is carried out through the classification module,and the required key information is structured through the post-processing step.In addition,this thesis optimizes the loss function to improve the accuracy of key information extraction.The proposed Medical examination report table recognition algorithm CoT＿SRN used TEDS as an evaluation metric.Compared with the excellent algorithms in recent years on the large public table recognition datasets Pub Tab Net and Sci TSR,it achieves competitive TEDS scores.This thesis also labeled and processed the Medical examination report table images collected in real medical scenarios.Made a Medical examination report table image datasets CMDD+ for the training and testing of experimental models,including ablation experiments,contrast experiments and cross-validation experiments.Experimental results show that the proposed algorithm achieves the highest TEDS score in the field of Medical examination report table recognition.In addition,ablation experiments were performed on the shopping receipt datasets Wildreceipt,the common ticket datasets CSIG＿Datasets,and the laboratory ticket datasets CMDD+.All the algorithms in this thesis achieve the highest accuracy. |
