Deep Learning For Caption Generation Of Ultrasound Image And Verification System

Caption generation of ultrasound image directly generates annotation text to describe the information for diseases and content in ultrasound image by algorithm.Inexperienced doctors and patients can directly use the annotation text to intuitively understand the information for diseases in the ultrasound image.The research for understanding of ultrasound image lacks of detailed analysis and automatic description.In order to easily,quickly,and delicately understand the information for disease in ultrasound images,this thesis introduces caption generation of image to ultrasound images,and proposes two types of caption generation of ultrasound image.The main contributions of this thesis are listed as follows:1.The research of ultrasound image understanding mainly focuses on the coarse-grained directions,such as retrieval,detection,classification,segmentation,etc.And the research lack of understanding the content of ultrasound images directly from the human language.This thesis proposes the grain-size classification for caption generation of ultrasound image by introducing caption generation of the image into the field of ultrasound image comprehension.The model utilizes the coarse classification model which composed by Convolutional Neural Networks(CNN)to recognize the organ of ultrasound image,and obtains an annotation text which descrides the information for diseases in the ultrasound image decoded by a Long Short-Term Memory(LSTM).The proposed model can be applied to the ultrasound image dataset constructed in this thesis.The experimental results show that when providing only training data for ultrasound images and diagnostic reports,the correct recognition rate of the ultrasound image by the encoder excess 90%,the LSTM language generation model also generates high-quality annotation text.2.A novel method is proposed for diseases detection of caption generation ultrasound image to describe the details of the lesion area of the ultrasound image,and eliminate interference of diseases between different organs in this thesis.The model simultaneously completes the detection and encoding of the area of diseases by detection model.Then the LSTM decodes the encoded vector to automatically annotation text to describe the information of diseases in ultrasound image.The proposed model is applied to the ultrasound image dataset constructed in this thesis.The experimental results show that when providing training data with diagnostic report and ground truth bounding boxes of the lesion area on the ultrasound image,the proposed model can reduce the interference of diseases between different organs,and improve 1% score on BLEU-1 and BLEU-2.The parameter amount is only 1/4 of the grain-size classification for caption generation of ultrasound image,and the operating efficiency is 1.4 times faster.3.This thesis designs and implements ultrasound image assisted understanding system.And integrates the two algorithm models proposed in this thesis into the system.The users load an ultrasound image into the system,and can automatically generate corresponding annotation text from ultrasound image through a simple interface operation.The system can facilitate the users to more easily understand the information of diseases in the ultrasound image,assist the doctors to diagnose the illness,and reduce the contradiction between doctors and patients.