Research On Localization Of Craniomaxillofacial Anatomical Landmarks Based On Deep Learning

Dento-maxillofacial deformities are a series of problems arising from the incongruous relationship between the teeth,jaws and face,mainly due to congenital genetic and acquired factors(e.g.tumors,trauma,etc.),resulting in misaligned teeth,malocclusion,abnormal size or position of the craniofacial bones and maxillo-mandibular abnormalities.It not only seriously affects one’s facial appearance,but may also induce diseases such as periodontal disease,dental caries,temporomandibular joint disorder,and speech dysfunction,and even cause psychological disorders.With the development of computer technology and imaging equipment,2D X-rays and 3D CBCT have become common images for the diagnosis and treatment of dental-maxillofacial deformities.In this process,the specialist needs to manually localize clinically significant anatomical landmarks for measuring and analyzing the degree of malformation and for planning surgical treatment.However,manual annotation is subjective,with intra-and inter-class variation in labeling,and wastes valuable time and effort of clinicians.An automatic,accurate,and reliable craniomaxillofacial landmark recognition and detection algorithm can solve these problems to some extent.This dissertation provides an in-depth study of deep learning-based landmark localization methods.It investigated automatic landmark detection on 2D X-rays at first.Based on this,it further studied automatic craniomaxillofacial landmark localization on 3D CBCT scans to accurately identify and reliably locate the position of each predefined anatomical landmark,which in turn assists clinicians in clinical diagnosis and treatment assessment,and furthermore in developing more accurate surgical plans.The details of the research in this dissertation are listed as follows:(1)This dissertation proposes a localization method based on appearance token and landmark token to detect landmarks on 2D X-rays accurately and reliably.The existing methods on 2D cephalograms still suffer from some limitations.First,the reasons like X-ray images with a large resolution and memory limitations oblige most methods to resize the input image which inevitably leads to down-sampling quantization errors.Second,preprocessing or postprocessing is required to improve localization accuracy and robustness.Third,landmarks are not independently located at the image and are constrained by others.To surmount these problems,it constructs an endto-end localization network.Firstly,image patches with fix size at different resolutions are sampled from the original image,and then multiscale features are extracted.Secondly,they are passed to linear projection and converted into appearance token,and input to the relational inference layer with landmark tokens,which makes the landmark tokens learn the inherent relationship between appearance tokens and landmarks to predict the relative displacements of multiple landmarks accurately.Finally,the model guides the initial points to move toward the targets in a cascade manner from coarse to fine after a few iterative reasoning steps.The experimental results demonstrate that the proposed method outperforms other advanced comparing methods in terms of mean radial error and successful detection rate within various precision ranges.A detection success rate of 82.38% was achieved within a clinically acceptable range of 2 mm,indicating that the method can be used for computerized cephalometry.(2)This dissertation develops a landmark localization algorithm using multiscale image patch-based graph convolutional networks(GCNs)to detect landmarks on 2D X-rays accurately and reliably.It proposes a cascaded localization method.First,the image patches with size of 64×64 at different scales are hierarchically extracted from the Gaussian pyramid centered on the current points,thus preserving the multiscale context information.Then,spatialized features are generated with an attention module that involve both local appearance features and shape information.The spatial relationships between landmarks are effectively established with three-layer GCNs,so that our model can utilize the learned structural knowledge to suppress the presence of false positives.The proposed approach has exhibited superior performance compared with other state-of-the-art methods in terms of mean radial error and successful detection rate.A successful detection rate of83.20% was achieved within a clinically acceptable range of 2 mm,indicating the potential of the method for application in cephalometric analysis and orthognathic surgery.(3)This dissertation presents a craniomaxillofacial landmark localization framework using geometric constraint and transformer,referred to as CMF-Net to detect anatomical landmarks on 3D CBCT scans accurately and reliably.2D X-ray images are commonly used in orthodontic diagnosis and treatment planning.However,3D CBCT scans can help clinicians accurately and reliably analyze complex upper and lower jaw and craniofacial deformities to improve the diagnosis and planning of orthognathic surgery.It proposes a trainable end-to-end localization framework,called CMF-Net,where the appearance branch integrates transformers for identifying the exact positions of candidates,while the geometric constraint branch at low resolution allows the implicit spatial relationships between these landmarks to be effectively learned.Adaptive wing loss is used to regress pixel values near the mode of the regressed volumetric heatmap.Comprehensive experiments on 150 dental CBCT scans show that the localization precision of the method is better than the current state-of-the-art deep learning methods,with an average localization error of 1.108mm(the clinically acceptable accuracy range being 1.5 mm)and a successful detection rate of 79.28%,demonstrating it can be applied to 3D cephalometric measurement,analysis and surgical planning.