Fine-grained Fetal Ultrasound Scan Planes Selection Based On Multi-label Joint Learning

Deep learning has achieved great success in many fields,such as speech recognition,natural language processing,and face recognition,but the research in the field of prenatal ultrasound image processing is still in the preliminary stage of exploration.In clinical practice,ultrasound technique is one of the main imaging modalities,which has the advantages of no radiation,low cost,and convenient operation.Therefore,prenatal examination of the fetus generally is to obtain the standard plane that can clearly show the key anatomical structures and provide an important basis for obstetricians to diagnose fetal diseases.At present,the fetal ultrasound standard plane is manually obtained by sonographers.Due to the difference in the position of the fetus and the influence of different gestational weeks,the position and feature of the key anatomical structures are also very different.It requires the obstetric sonographer to have a lot of anatomical knowledge and rich clinical experience,which is a labor-intensive and highly technical work.With the rapid development of artificial intelligence technology and the broad applications of prenatal ultrasound medical application,the research of deep learning in the field of ultrasound image processing has become an emerging hot spot,and it has also brought many challenges.This article proposes a one-stop end-to-end computer-assisted solution for the automatic identification of six ultrasound standard planes of fetus.In order to reflect the representativeness of the whole fetal ultrasound standard recognition scene,these six kinds of standard planes contain the part standard planes involved in primary,secondary and tertiary obstetric ultrasound examinations.This application scene has the following characteristics: while distinguishing the plane categories,each plane can be divided into three categories: standard,basic standard and non-standard;There is the high intra-class and low inter-class variations of standard planes.Therefore,this article proposes a fine-grained prenatal ultrasound image recognition algorithm based on multi-label joint learning which named FGMixed-RCNN.The main contribution are summarized as follows.(1)We solve the issues of multi-category labeling and coarse-grained &fine-grained mixed classification in one ultrasound image.Based on the classic two-stage object detector,i.e.,Faster-RCNN,this article designs a novel network architecture and proposes a hierarchical multi-label algorithm.It was designed as a multi-task joint learning algorithm,which can separate coarse-grained task and fine-grained task perfectly.(2)Aiming at the issue of fine-grained classification in a specific category,this article proposes to design a novel network branch which named 133 SE.The 133 SE branch includes a Squeeze-Excitation structure which attentions to the relationship of features in the channel dimension of feature maps.In addition,we introduce an additive angular margin loss and cross-entropy loss function for the 133 SE branch task to further improve the classification performance.(3)The proposed method is evaluated on a large-scale ultrasound datasets,i.e.,48000 samples.Experimental results show that the proposed algorithm obtains good performance,which provides the possibility for its clinical application and promotion to other detection tasks.