Automatic Generation Of Thyroid Ultrasound Image Report Assisted By Thisroty Report

Medical imaging has now become the key basis for early screening,differential diagnosis and treatment of various clinical diseases.With the rapid development of medical imaging technology and the rapid increase of image data,clinical and imaging doctors are facing huge diagnostic pressure and report writing workload.For a given medical image,how to automatically identify the content of the image,extract key information,and further generate an image report with complete information and standardized structure is of great significance.In recent years,artificial intelligence,especially deep learning technology,has made great progress in tasks such as natural language processing and image recognition.Applying deep learning to the medical field,research on automatic generation of image reports based on deep learning has become a current research hotspot.The image report generation can be divided into two steps,image understanding and natural language text generation.Therefore,most of the existing image report generation based on deep learning uses an encoder-decoder structure to model the above steps.Encoder extract image features or keyword information,and decoder generates corresponding report based on the extracted features or keyword information.However,the content complexity and long text characteristics of medical imaging pose huge challenges for the above two steps.The diagnosis report is usually composed of several small paragraphs describing the results of the imaging observations in detail.Existing model methods are difficult to model very long sequences.The accuracy and fluency of generating reports are low.In addition,most of the existing encoders potentially learn the relationship between image pixels and keywords in the report,but medical images themselves are highly complex,and certain attributes of medical images are prone to ambiguity during observation.The above-mentioned factors lead to the low quality of the current reports and limited clinical significance.In order to solve the above problems,this article introduces historical reports in the process of automatic report generation.With the help of the structure and content of historical reports,the quality of current reports is improved.The main work of this paper is as follows:1)Propose a method for automatically generating image reports assisted by historical reports.Design a dual encoder-decoder structure with an attention mechanism for report generation.The dual encoder separately encodes historical reports and image keywords and extracts text structure information and potential semantic information in image keywords and historical reports.Aiming at the structure of the dual encoders,a joint attention mechanism module is designed,which can dynamically learn the information extracted by the two encoders.Finally,an image inspection report is generated by the decoder.2)Propose an improved model for the automatic generation of image reports that integrate image interpretation.In the former method,the focus is on how to use the structure and semantic content of the historical report.The input of the image keyword module is the keyword information extracted from the image.In further work,focusing on the image interpretation part,a keyword extraction algorithm based on the thyroid ultrasound semantic tree is proposed to provide a reference for the definition of image semantic tags.On this basis,the model structure and classification process of multi-label classification of thyroid nodules images are introduced.Furthermore,the image interpretation and the report generation model assisted by historical reports are combined to explore the mutual promotion mechanism of historical reports and image interpretation.3)Use actual ultrasound report data to verify the model.This article uses the thyroid ultrasound report data set(desensitized,containing only text information)of a Shanghai tertiary hospital to verify the effectiveness of the proposed algorithm.Analyze the effectiveness of the image report automatic generation model assisted by historical reports from three aspects: evaluation indicators,sample analysis,and attention hotspot distribution maps.Experimental results prove that the model can make full use of the text structure information and potential semantic information of historical reports to generate reports with accurate content,rich semantic and smooth sentences.The effectiveness of the image report automatic generation model is analyzed from the two aspects of image classification effect and report generation effect.The experimental results prove that the model can jointly learn image features and text features and promote each other to improve the quality of image interpretation and report generation.