Nomogram Combining Ultrasound-based Radiomics With The Risk Classification System For Diagnosing Malignancy In Thyroid Nodules

Part Ⅰ Accuracy of Thyroid Imaging Reporting and Data Systems for diagnosing malignancy in thyroid nodulesObjectives:Ultrasound is the most accurate imaging tool for evaluating thyroid nodules,and several authoritative societies have developed risk-stratification systems based on B-mode ultrasound（BMUS）images.This study aimed to compare diagnostic accuracy and biopsy rate of four risk-stratification systems sonographic thyroid risk stratification guidelines,including the Kwak Thyroid Imaging Reporting and Data System（Kwak TI-RADS）,American Thyroid Association（ATA）Guideline,European Thyroid Association（EU）TI-RADS and American College of Radiology（ACR）TI-RADS,for diagnosing malignant thyroid nodules.Methods:1.This study included 451 thyroid nodules with a maximum diameter greater than or equal to 10 mm in the First Affiliated Hospital of Nanchang University from January 2019 to September 2020.2.To apply Kwak TI-RADS,ATA,EU TI-RADS and ACR TI-RADS to classify benign and malignant thyroid nodules and provide biopsy recommendations by two sonographers with 3 years and more than 10 years of work experience respectively.3.To evaluate the relationship between each grading of the 4 risk-stratification systems and nodule pathology results.4.To compare the sensitivity,specificity,negative predictive value,accuracy and area under the receiver operator characteristic curve（AUC）of the 4 risk-stratification systems for diagnosing malignant nodules.5.For ATA and ACR TI-RADS,which are giving fine-needle aspiration（FNA）recommendations,this study compared unnecessary FNA rates.Results:1.The proportion of malignant nodules in the 451 nodules included was 65.2%（294/451）.There was significant difference in the 5 types of ultrasound features（composition,echogenicity,shape,margin,and calcification）evaluated by the Kwak TI-RADS,ATA,EU TI-RADS and ACR TI-RADS between benign and malignant nodules（all p<0.05）.2.The 4 risk-stratification systems applied by sonographers with different proficiency experience were all significantly correlated with pathological results of thyroid nodules,among which the ACR TI-RADS had the highest correlation coefficient（senior physicians:0.505,junior physicians:0.459;both p<0.001）.3.For both senior and junior sonographers,ACR TI-RADS had the highest sensitivity,negative predictive value and accuracy for diagnosing malignant thyroid nodules.For the senior sonographer,the Kwak TI-RADS had the highest specificity and positive predictive value,while the EU TI-RADS had the highest specificity and positive predictive value for the junior sonographer.4.Compared with the Kwak TI-RADS,ATA guideline and EU TI-RADS,the ACR TI-RADS had the highest AUC（0.814,95%CI 0.770-0.857）for the senior sonographer and the highest AUC（0.786,95%CI 0.741-0.830）for the junior sonographer,which were significantly higher than AUCs of the Kwak TI-RADS,ATA guideline and EU TI-RADS（all p<0.05）.5.The unnecessary FNA rates of ACR TI-RADS were lower than that of ATA guideline by sonographers of different proficiency experience（senior sonographer:20.25%vs.29.16%,junior sonographer:26.45%vs.34.30%）.Conclusions:The ACR TI-RADS by sonographers with different proficiency experience can significantly improve the diagnostic accuracy for malignant thyroid nodules compared with other risk-stratification systems and reduce the unnecessary FNA rate;therefore,the ACR TI-RADS is a recommended risk-stratification system.Part II Nomogram Combining Radiomics with the American College of Radiology（ACR）Thyroid Imaging Reporting and Data System（TI-RADS）for diagnosing malignancy in thyroid nodulesObjectives:Radiomics can mine quantitative image features that cannot be assessed by visual interpretation,which is an imaging tool with great potential in tumor diagnosis.This study aimed to construct a radiomic score（Rad-Score）based on B-mode ultrasound（BMUS）images,and to develop and validate a nomogram combining radiomics of B-mode ultrasound（BMUS）images and the American College of Radiology（ACR）Thyroid Imaging Reporting and Data System（TI-RADS）for predicting malignant thyroid nodules and improving the performance of the guideline.Methods:1.This study included 451 thyroid nodules with a maximum diameter greater than or equal to 10 mm verified by pathology from January 2019 to September 2020 in the First Affiliated Hospital of Nanchang University and were randomly divided into the training cohort（n=315）and validation cohort（n=136）according to the ratio of 7:3.2.To apply the 3D Slicer to delineate the region of interest（ROI）on the BMUS image of the largest diameter section of the thyroid nodule,and perform feature extraction by Pyradiomics.3.To develop a Rad-Score based on radiomic features of thyroid nodules in the training cohort,and verify in the validation cohort.Discrimination was assessed using the rea under the receiver operator characteristic curve（AUC）.4.The Rad-Score in the training cohort and the average ACR TI-RADS score（ACR-score）of sonographers with 3 years and more than 10 years of work experience respectively were combined to construct a radiomic nomogram（ACR-Rad nomogram）and verified in the validation cohort.Discrimination was assessed using AUC,index integrated discrimination improvement（IDI）and net reclassification improvement（NRI）.Calibration was assessed using calibration curves and the Hosmer-Lemeshow test.Clinical utility was assessed using a clinical decision curve analysis（DCA）.5.To calculate the unnecessary fine-needle aspiration（FNA）rate of radiomic nomogram malignancy risk cutoffs based on 20%/30%/40%/50%and maximum Youden index（referred to as ACR-rad₂0%/30%/40%/50%/max,respectively）,and compare with the ACR TI-RADS.Results:1.There were no significant differences in clinical,pathological and ultrasound characteristics between the training cohort and the validation cohort（p>0.05）.The proportions of malignant nodules in the two groups were 62.9%（198/315）and 70.6%（96/136）（p=0.114）.In the training cohort,the Rad-Score of malignant nodules was significantly higher than that of benign nodules（1.265[0.738-1.900]vs.-0.005[-1.955-0.910],p<0.001）,and the difference of the Rad-Score in the validation cohort was still significant（1.177[0.355-1.845]vs.-0.320[-2.182-0.685],p<0.001）.2.The Rad-Score had good diagnostic performance,with an AUC of 0.801（95%CI 0.750-0.851）in the training cohort and 0.820（95%CI 0.742-0.898）in the validation cohort.3.In the training cohort and the validation cohort,the AUC of the ACR-Rad nomogram were 0.877（95%CI 0.836-0.919）and 0.864（95%CI 0.799-0.931）,respectively,which were significantly higher than that of the ACR-score（p<0.001 and 0.031）.In both cohorts,the nomogram had good calibration performance,and DCA showed that the nomogram has clinical application value.4.AUC of the ACR-Rad nomogram were significantly higher than that of the ACR TI-RADS score evaluated by senior and junior sonographers,respectively.Both NRI and IDI showed that the nomogram had a significant improvement in the diagnosis malignant nodules by sonographers with different proficiency experience（p<0.001）.5.When applying ACR-rad₅0%to recommend FNA,the ACR-Rad nomogram showed higher specificity,accuracy and positive predictive value compared to ACR TI-RADS and reduced the unnecessary FNA rate.Conclusions:Ultrasound-based radiomics was an accurate tool for diagnosing benign and malignant thyroid nodules.The ACR-Rad nomogram,combining with ACR TI-RADS and BMUS radiomics,has the potential to be a convenient and accurate tool to predict malignancy and improve performance for radiologists at different proficiency levels in thyroid nodules.