Research On Medical Image Report Generation Method Based On Deep Learning

Medical image is widely used in clinical diagnosis and treatment.However,the writing of medical image diagnostic reports is a complicated task.For inexperienced doctors,it is easy to make mistakes in writing reports;for experienced doctors,writing reports is time-consuming and tedious.With the application of artificial intelligence technology and deep learning methods in the medical field,researchers propose some methods of generating medical image reports based on deep learning,which brings convenience to doctors and improves work efficiency.However,there are also various difficulties in this task.First,the medical image report generation task needs to generate a description based on a given medical image.The large number of words and the transfer association and semantic opposition between sentences in the report,which greatly increases the difficulty for the processing of this task and the measurement of the generated results.Second,each region of the medical image corresponds to a description.In response to the above problems,this paper fully studies the current research findings in the field of medical image report generation task at home and abroad,and proposes two medical image report generation models based on generative adversarial network.The work contents are as follows.(1)Proposing the Medical Image Report Generative Adversarial Network(MIRGAN).The generator adopts the encoder-decoder network structure.The discriminator takes each sentence of the report as input and its output as a kind of "feedback" to guide the update of the generator.When the generative adversarial network deals with discrete character sequences,the generator cannot handle the gradient update from the discriminator.Thus the idea of reinforcement learning is used to enable the discriminator to pass "reward" to guide the update of the generator.(2)Proposing the With Two Rewards Generative Adversarial Network(W2RGAN).Based on the MIRGAN model,the W2 RGAN model improves the reward mechanism.A hybrid discriminator model is proposed,and two rewards are given,namely,one-sentence reward and one-word reward.And these two rewards learn the structure information of the sentence and the word diversity information respectively.Through the combination of two rewards,the generator gets better "guidance" and improves performance.(3)Experiments are carried out on the IU X-Ray to verify the effectiveness of the two models proposed in this paper in the task of medical image report generation.The results of qualitative and quantitative experiments show that the two models proposed in this paper have better performance and can generate higher quality medical image reports.