Research On Medical Image Report Generation Method Based On Multi-view Fusion

As the core carrier of recording image content and diagnostic information,medical image report is an important medium for communication between clinicians and radiologists.With the increasing demand for medical image examination,the traditional manual interpretation method has been unable to cope with the massive medical image data,resulting in a series of problems such as the prolongation of the patient’s medical treatment cycle and the aggravation of the doctor’s workload.Therefore,the development of intelligent medical image report generation holds significant implications for alleviating radiologists’ workloads and enhancing the quality of healthcare services.With the update and iteration of medical imaging technology,the original single modality,single view and static medical images have gradually developed to multi-modal,three-dimensional and dynamic.However,most of the current research on medical image report generation only focuses on single modality,single view and static medical images,lacking consideration of multi-modal,multi-view and dynamic medical image data,which limits the application of medical report generation technology.Based on the generation of image reports for multi-modal,multi-view and dynamic medical image data,thesis has carried out the following studies around the three key issues of feature extraction,feature fusion and report generation.Research on medical image information extraction algorithm based on multi-view fusion.In response to the challenge of extracting information from multi-view medical images,thesis proposes a multi-view feature fusion module that utilizes channel attention mechanisms and three-dimensional convolution to extract and deeply fuse multi-view features,thereby improving the multi-view feature extraction and fusion capabilities of existing feature extraction networks,and providing accurate and semantically consistent visual information for the report generation network.Research on medical image report generation based on multi-view and multi-modal fusion.A multi-view and multi-modal feature fusion module is designed for extracting and deeply fusing multi-view and multi-modal features.On this basis,a medical image report generation network based on multi-view and multi-modal fusion is proposed,and accurate and complete image reports are generated for multi-view and multi-modal medical images through the hierarchical LSTM model based on joint attention.Research on medical image report generation algorithm based on multi-view and temporal feature fusion.Thesis adds temporal feature fusion on the basis of the multiview feature fusion module,deeply fuses the multi-view feature and temporal feature through self-attention and cross-modal attention focus,and then proposes a medical image report generation model based on the multi-view and temporal feature,which generates accurate and complete image reports for temporal image data.