Research On The Application Of Deep Learning In Prenatal Ultrasound Standard Views Classification,Visualization And Temporal Learning

In prenatal ultrasound diagnosis,the detection of fetal standard views(FSVs)is a critical step,which is a precondition and essential step for subsequent structural observation,parameter measurement and abnormal diagnosis.The detection of FSVs is time-consuming,tedious and heavily relies on the experience of the operator.It can be difficult for inexperienced doctors to pinpoint FSVs,which increases the risk of missed diagnosis.Therefore,it is of great significance to design an automatic FSVs classification system for reducing subjective bias,improving the efficiency of screening and reducing missed detection rate.Compared with other imaging methods,ultrasound is often affected by noise,distortion,and artifacts.These issues result in poor image quality and bring more challenges.Meanwhile,ultrasonic tasks require high real-time performance of the intelligent algorithm.Due to the continuity of the ultrasound scan views,it is difficult to select several standard views with the only subtle difference between the standard image and the non-standard image from adjacent frames.Therefore,it is a highly challenging task to design an efficient real-time FSVs classification and detection system.To address these issues,we start from three aspects,with the help of the deep learning method to study the problem of FSVs detection and propose corresponding novel solutions.1.Real-time Classification of FSVs in Prenatal Ultrasound.In practice,the intelligent ultrasonic system needs to be able to run in real-time,but GPU computing resources on hospital equipment are less.Few previous studies have considered algorithm speed and computational complexity in model design.In this part we mainly take the real-time as the entry point,and use the multi-scale dense network to design a real-time FSVs classification system.Its multiscale and dense connection combine the fine level features from the shallow layers and coarse level features from the deep layers,which make network learning more efficient.Also,it adopts cascade design to make the network depth can be selected according to the task difficulty,which further economize computation.2.Visualization & Weak Supervised Localization.In the task of deep learning,especially in medical image analysis,the interpretability and persuasiveness of the system are weak,when only the final prediction results are given.We want to know not only about the network’s decisions but also the basis on which they are made.In this part,we use the class activation mapping method to study the interpretability of neural networks in FSVs classification task,analyze the relationship between the visual focus of network and the classification result,and study the attention mechanism of the network by heat map.Finally,we propose a weakly supervised localization scheme based on the adversarial erasing strategy.3.Research on FSVs Detection and Temporal Learning Method.Ultrasound video is not only a collection of individual frames but also be rich in temporal information between frames.In this part,we add inter-frame temporal information in learning models and perform methodological research on several potential temporal learning architectures in the frame-level detection task,including frameworks based on optical flow,LSTM(Long Short-Term Memory),and 3D convolutional networks.Then we design a novel Frame-level 3D Convolutional Network(F3D).Its unique multi-stage upsampling mode enables the network to produce dense prediction results of each frame.Finally,our scheme achieved the real-time detection of FSVs in ultrasound video.