Automatic Generation Of Medical Imaging Report Based On Deep Learning

Medical imaging data contains a wealth of health information,which is the key basis for early screening,differential diagnosis and treatment of a variety of clinical diseases.With the rapid development of medical imaging technology,medical imaging data increases exponentially,which brings great diagnostic pressure and report writing load to clinicians and radiologists.How to effectively extract valuable information from medical images to assist imaging diagnosis and report writing is a great challenge.In recent years,with the rapid development of artificial intelligence technology,the application of deep learning in the field of medicine has greatly increased,which drives a large number of researches on automatic generation of medical imaging reports based on deep learning.The current research on medical imaging report generation has achieved some success,but there are still many limitations,which do not fully combine the medical and semantic features of the medical imaging report but simply train models to forcibly align the image features with the report text features.Thus the quality of the generated report is not high and the clinical significance is limited.Based on this,this study proposes a new medical imaging report generation model based on template and multi-attention mechanism.The model fully combines the medical and semantic features of the medical imaging report and the advantages of deep learning,thus realizes the automatic generation of medical imaging report more accurately and effectively.The main contents of this study include:(1)We investigate the current open accessible chest imaging datasets,analyze the structure,content and characteristics of each medical imaging dataset,and then select the most appropriate dataset to support the construction and training of our medical imaging report generation model.We review literature on researches of chest lesion recognition and medical imaging report generation based on medical imaging data,and then systematically comb the advantages and limitations of previous studies,which provides a sufficient theoretical basis for the construction of new research method.(2)In this study,a new medical imaging report generation model(TMRGM)based on template and multi-attention mechanism is proposed.The TMRGM model fully combines the medical characteristics of the medical imaging report during the medical imaging report generation,and adopts different methods to generate the medical imaging reports for the healthy population and the sick population.In the generation of medical imaging reports for healthy people,an medical imaging report template library,containing 63 template sentences about the Impression field and 150 template sentences about the Findings field,is constructed by manually tagging and screening.Six sentences of the report template library are selected to form the medical imaging report template for generating the medical imaging report of healthy people.On the generation of imaging reports of sick people,by adding Co-attention and Adaptive attention mechanism to our model,we effectively fuse the image features and the text features of the lesion tags and make the model automatically select whether to generate the medical imaging report based on image features,sentence topics or generated text features.Finall,we realize high quality medical imaging report generation.(3)We train and evaluate the proposed TMRGM model based on the OpenI dataset.In this study,accuracy,recall and F1 score are used to evaluate the accuracy of TMRGM model in chest lesion recognition,and BLEU score,METEOR,ROUGE and CIDEr are used to evaluate the effect of TMRGM model on medical imaging report generation.The experimental results show that the accuracy of TMRGM model is better than the baseline models such as TieNet,Adapt-att and Co-att(BLEU-1 is 0.419,METEOR is 0.183,ROUGE score is 0.280,CIDEr is 0.359),which proves that the TMRGM model is accurate and effective in medical imaging report generation.To sum up,in this study,based on reviewing the existing chest imaging datasets as well as the researches of medical imaging report generation,we propose a new medical imaging report generation model based on template and multi-attention mechanism.Then we verify the accuracy and the effectiveness of the model on the OpenI chest X-ray dataset.It is hoped that the proposed model can assist radiologists to quickly find out the chest lesions and complete the writing of medical imaging reports,thus can reduce the burden of film reading and report writing,improve the accuracy of imaging diagnosis and the quality of the medical imaging report,and finally realize better diagnosis and treatment of patients.