The Development Of Deep Learning Based System For Intelligent Analysis Of The Lumbar Spine MR Images

Part 1: A deep learning based program Spine Explorer(Yitian)for automated analysis of sagittal lumbar MRIBackground Although quantitative measurements could improve the assessment for disc degeneration,acquirement of such quantification results still relies on manual segmentation on lumbar magnetic resonance images(MRI),which is a time-consuming process and may introduce subjective bias.To date,however,only a few semiautomatic methods have been developed to quantify major components on spine MRI.Objective To develop a deep learning based program(Spine Explorer,Yitian)for automated segmentation and quantification of the vertebrae and intervertebral discs on sagittal lumbar spine MRI.Methods The study was extended on the Hangzhou Lumbar Spine Study,a population-based study of mainland Chinese with focuses on lumbar degenerative changes.From this population-based database,50 sets lumbar MRI were randomly selected as training dataset,and another 50 as test dataset.The vertebrae and discs were manually segmented on T2 W sagittal MRI to train a convolutional neural network,UNet for automatedsegmentation.Intersection-over-union was calculated to evaluate segmentation accuracy.Computational definitions were further proposed to acquire quantitative morphometric and signal measurements for both lumbar vertebrae and discs.MRI in the test dataset were automatically measured with Spine Explorer(Yitian)and manually with Image J.Intra-class correlation coefficient(ICC)were calculated to examine inter-software agreements.Correlations between disc measurements and Pfirrmann score as well as age were examined to evaluate measurement validity.Results The trained Spine Explorer(Yitian)automatically segments and measures a lumbar MRI in half a second,with mean segmentation Intersection-over-union of 94.7% and 92.6% for the vertebra and disc,respectively.For both vertebra and disc measurements acquired with Yitian and Image J,the agreements were excellent(ICC=0.81~1.00).Disc measurements significantly correlated to Pfirrmann score,and greater age was associated with greater anterior disc bulging area(r=0.35~0.44)and fewer signal measurements(r=0.62~0.77)as automatically acquired with Ytian.Conclusions Spine Explorer(Yitian)is an efficient,accurate,and reliable tool to acquire comprehensive quantitative measurements for lumbar vertebra and disc.Implication of such deep learning based program can facilitate clinical studies of the lumbar spine.Part 2: A deep learning based program Spine Explore(Tulong)for automated analysis of axial lumbar MRIBackground Paraspinal muscles have been extensively studied on axial lumbar magnetic resonance images(MRI)for better understanding of back pain;however,the acquirement of measurements for the paraspinal muscles mainly relies on manual segmentation,which is time consuming and may introduce subjective bias.Objective To develop and validate a deep-learning–based program(Spine Explorer,Tulong)for automated acquisition of quantitative measurements for paraspinal muscles and other major spinal components on axial lumbar MRI.Methods T2-weighted axial MRI at the L4-5 lumbar disc level of 120 subjects were selected from the Hangzhou Lumbar Spine Study database,to develop and validate the deeplearning–based program Spine Explorer(Tulong).90 axial MRI were randomly selected as training set,and the other 30 were regarded as test set.Axial lumbar MRI in the test set were automatically measured by Tulong and then manually measured using Image J to acquire quantitative size and compositional measurements for bilateral multifidus,erector spinae,and psoas muscles,as well as the disc and spinal canal.Dice coefficient were used to evaluate the performance of automated segmentation.Intra-class coefficients(ICCs)and Bland-Altman plots were used to examine inter-software agreements for various measurements.Results After training,Tulong could measure an axial lumbar MRI in less than 1 second.The Dice coefficient were 83.3% to 88.4% for multiple paraspinal muscles,92.2% and 82.1% for the disc and spinal canal,respectively.For various quantitative measurements in thesize and compositional of paraspinal muscles,automated Tulong was in good agreement with manual Image J(ICC = 0.85~0.99).Conclusions Spine Explorer(Tulong)is efficient and reliable for the quantifications of multiple paraspinal muscles,disc and canal.Various size and compositional measurements could be simultaneously obtained for the lumbar paraspinal muscles through the developed software.Such fully automated program might encourage further epidemiological studies of the lumbar paraspinal muscle degeneration and improve paraspinal muscle assessment in clinical practice.Part 3: Segmentation of paraspinal muscles at varied lumbar spinal levels by Explicit Saliency-Aware LearningBackground Automated segmentation for paraspinal muscles on axial lumbar MRI of varied spinal levels is clinically demanded.However,it is a challenging task and there is no reported success due to the large inter-and intra-organ variations,unclear muscle boundaries and unpredictable muscle degeneration patterns.Objective To develop and validate a novel deep learning algorithm,thus,achieve precise segmentation for multiple paraspinal muscles on axial MRI at varied spine levels.Methods In this paper,we propose a novel explicit saliency-aware learning framework(BSESNet)for fine segmentation of multiple paraspinal muscles and other major components at varied spinal levels across the full lumbar spine.BS-ESNet is designed to first detect the location of each organ in forms of bounding box(b-box);then performs accurately segmentation which utilizes detected b-boxes to enable spatial saliency awareness.BSESNet creatively conducts detection upon a preliminary segmentation mask instead of input MRI,which eliminates the influence of inter-organ variations and is robust against unclear muscle boundaries.Such segment-then-detect workflow also provides a paradigm to formulate multi-organ detection in an end-to-end trainable process.Our framework also embeds an elaborate spatial attention gate(SAG)which adopts detection b-boxes to obtain a saliency activation map in an explicitly supervised manner.Acquired salient attention map can automatically correct and enhance segmentation features,and further guides the adaptation of variable precise anatomical structures.The newly proposed algorithm is validated on a challenging dataset,which contains 320 axial MRI from L2/3 to L5/S1 disc level.Standard five-fold cross-validation was performed for evaluation,and Dice coefficient(Dice)was used for the evaluation ofsegmentation performance.Ablation experiments was conducted to validate the effectiveness of SAG module,and BS-ESNet was further compared with other state-ofthe-art algorithms.Results Ablation study confirmed that the segmentation improvement of BS-ESNet comes from our elaborate designed SAG module.BS-ESNet achieves excellent segmentation accuracy for each targeted muscle and spinal component,with a mean Dice of 94.4%,and outperforms other stateof-the-art deep learning frameworks.Conclusions For the first time,this paper proposed an explicit saliency-aware learning framework for segmentation of the paraspinal muscles on axial lumbar MRI at multiple lumbar spinal levels.The extensive experiments revealed that our proposed BS-ESNet is reliable and accurate,and outperforms other state-of-the-art methods,suggesting it can be generalized in epidemiological studies of the lumbar spine.