Accuracy Of An Artificial Intelligence Multi-class Algorithm In The Diagnosis Of Common Skin Diseases

BackgroundMost artificial intelligence(AI)algorithms developed in dermatology were aimed at binary or multi-class classification of skin tumors.Literatures in multi-class AI algorithms for diagnosis of neoplastic,inflammatory and infectious skin diseases were limited.The multi-class AI algorithm tested in this study was able to identify 44 common skin diseases.ObjectivesTo validate the diagnostic accuracy of a multi-class AI algorithm and to evaluate its value in diagnosis of common skin diseases.MethodsFrom February 2021 to August 2021,761 skin diseases patients were prospectively recruited from department of dermatology of the First Affiliated Hospital of Zhengzhou University.A single clinical image of each patient was chosen and uploaded onto the application for diagnosis.The algorithm’s diagnostic accuracy indices,including sensitivity,specificity,accuracy and kappa value,were calculated for all the patients with in-distribution disease classes.30 images of non-human skin objectives(NHSO),20 images of normal human skin(NHS)from 20 volunteers,normal human skins wrapped with a few thin materials,and normal human skin with stains from 3 volunteers,were submitted to the algorithm for its outputs.ResultsThe algorithm output five diagnoses for each of these 761 cases.In the diagnosis of all of the in-distribution skin diseases,it achieved mean top-1/top-3 sensitivity of 53.4%(95%CI 36.4-70.4)/76.9%(95%CI 62.4-91.4);mean top-1/top-3 specificity of 96.3%(95%CI 90.8-100.0)/85.7%(95%CI 68.0-100.0);and mean top-1/top-3 accuracy of 92.3%(95%CI 86.3-98.2)/84.7%(95%CI 68.4-100.0%),respectively.The overall agreements between the AI’s top-1/top-3 diagnosis and that of the dermatologists were moderate(k=0.473,95%CI 0.264-0.681/k=0.477,95%CI 0.187-0.766),respectively.This algorithm also output 5 diagnoses for an image of:a.NHS;b.NHS with certain stains or covered by certain thin materials;c.some human-skin imitators.It correctly recognized most NHSOs with an output of "no skin detected in the image";but took some human-skin imitators for human skin.ConclusionsThis AI algorithm has demonstrated overall high accuracy,moderate sensitivity,high specificity in the diagnosis of common skin diseases.It is noteworthy that it may take human-skin imitators for skin or take stains over skin as skin lesions,and may output 5 diagnoses for whatever it perceives as "skin" in an image,including normal human skin,out-of-distribution skin disease(ODSD)and some human-skin imitators.