The Effect Of Interobserver Agreement On The Diagnostic Efficacy Of Endoscopic Ultrasound In Pancreatic Lesions

Objective Endoscopic ultrasound(EUS)is a safe and well-tolerated diagnostic method with high spatial resolution and real-time imaging advantages,and has been recommended for the diagnosis and treatment of pancreatic lesions.Moreover,the pathological diagnosis of pancreatic lesions can be obtained through EUS-guided fine needle aspiration(EUS-FNA),which has a significant impact on improving the overall diagnostic efficiency of EUS.However,many factors affect the diagnostic accuracy of EUS and EUS-FNA for pancreatic lesions,among which the EUS examination and cytological evaluation of EUS-FNA specimens are highly relevant to the EUS practitioner and cytopathologist,respectively.The evaluation and improvement of interobserver agreement(IOA)of EUS physicians and cytopathologists,and the standardization of diagnostic procedures of EUS and EUS-FNA are particularly important in improving the efficacy and quality control of EUS for the diagnosis and treatment of pancreatic diseases.However,IOA-related data and studies on cytopathologists and endoscopists are still very lacking.In this study,the agreement of these two types of observers in the diagnosis of pancreatic lesions by EUS was analyzed and evaluated,in order to further improve the diagnosis and treatment efficiency and quality control level of pancreatic lesions by EUS.Methods 1.The patients who received EUS-FNA for pancreatic solid lesions(SPLs)from October 2017 to 2021 were continuously collected.Their demographic data,past medical history,clinical symptoms and laboratory indicators were recorded,and the original EUS reports of all patients were reviewed.EUS data such as lesion location,size,boundary,echo characteristics,bile duct dilatation,pancreatic duct dilatation,peripancreatic lymph nodes,vascular invasion,and puncture frequency were extracted,and the original cytological results and final clinical diagnosis must be clearly marked.A standardized cytological specimen scoring tool was introduced,including a variety of quantitative indicators(number of nucleated cells and number of diagnostic cells)and quality indicators(blood turbidity,gastrointestinal contaminants,and preparation and staining artifacts).Cytological specimens of EUS-FNA were reordered from all patients by two cytopathologists with extensive experience in pancreatic cytology.The cytological diagnosis was rediagnosed and scored according to the above standardized assessment tools.The IOA of individual indicators and cytological diagnosis of EUS-FNA specimens were further analyzed by cytopathologists,especially the false-negative cases of cytological diagnosis(those who were cytologically diagnosed as benign lesions and eventually clinically diagnosed as malignant tumors).The regression model of cytological false-negative occurrence was constructed using clinical and EUS indicators.,then analyzed the influencing factors.2.Patients who visited our hospital from October 2017 to December 2022 and received EUS examination for identifying pancreatic lesions were collected.Basic information(gender,age)and baseline EUS parameters of all patients were recorded.Patients with pancreatic cystic diseases(PCLs)with definite pathological diagnosis were further included,and clinical data,laboratory indicators and imaging results were collected.For patients who received EUS-FNA,biochemical analyses including amylase,CEA,CA19-9,liquid-based cytology and histology results of the cyst fluid should also be collected,while for patients undergoing surgical resection,definitive pathological findings of the surgical specimen should be labeled.EUS static images of the patients with PCLs were independently observed by three EUS physicians,and the morphological characteristics of PCLs were re-evaluated and recorded,including basic morphology,cyst wall,cyst cavity contents,the relationship between lesions and pancreatic duct,and peripancreatic lymph nodes.Consistency tests were performed on individual morphological features of PCLs evaluated by EUS physicians,and the accuracy of endoscopic diagnosis was calculated according to the final clinical diagnosis of patients.Results 1.Finally,161 patients with SPLs were included.Their primitive cytological findings were classified as 54 cases of benign(34%),28 cases of atypical(17%),30cases of suspect malignant(19%)and 49 cases of malignant(30%).Matching the cytological diagnosis of EUS-FNA with the final clinical diagnosis,the"benign"and"atypical"were regarded as negative results,while"suspected malignant"and"malignant"were regarded as positive results.60 cases of false-negative cytological diagnosis were identified.Comparison of clinical and laboratory parameters between the two groups showed statistically significant differences in platelet counts and CEA levels(z=-2.203,p=0.028;z=-2.454,p=0.014),only the difference in lesion size was statistically significant in EUS indicators(χ~2=5.552,p=0.018).In IOA analysis of EUS-FNA samples assessed by cytopathologists,the IOA of cytological diagnosis of all cases was very perfect(Kendall’s W=0.896,P<0.05),while the false-negative group had only moderate agreement(Kendall’s W=0.462,P<0.05).After the original cytological diagnosis was excluded,the IOA of cytological diagnosis for all cases and the false-negative group were decreased by two re-evaluated cytologists,with weighted kappa values of 0.773 and 0.399,respectively.IOA analysis of single quantitative and qualitative indicators in standardized scoring tools showed that the quantitative indicators of all cases had significant agreement in the number of diagnostic cells(weighted kappa,κ=0.721),but the number of nucleated cells had poor agreement(κ=0.391),and the agreement of all qualitative indicators was moderate.In the false-negative group,except for the poor agreement of gastrointestinal pollutants(kappa=0.302),other qualitative and quantitative indicators only had moderate agreement.A binary logistic regression analysis model for false-negative cytological diagnosis was constructed.The results showed that pancreatitis,the size of the lesion,pancreatic duct dilatation,bile duct dilatation,number of puncture and other factors were not significantly correlated with the occurrence of false-negative diagnosis.However,as the number of diagnostic cells increased,the probability of false-negative diagnosis decreased(OR=0.337,p<0.05),and the possibility of false-negative diagnosis was significantly increased in the presence of blood turbidity(OR=3.121,p<0.05).In addition,according to different cytopathological grading criteria,two uncertain cytological diagnosis categories of atypical and suspected malignancy were included into the positive criteria successively.The results showed that when"atypical or suspected malignancy or malignant"were regarded as diagnostic criteria,EUS-FNA had the highest diagnostic efficacy in pancreatic cancer with 66.46%(107/161)of diagnostic accuracy rate.The sensitivity and specificity was 76%and 91%,respectively.2.A total of 1025 pancreatic EUS examinations were completed during the study.If the same patient underwent multiple EUS examinations,only their first examination or puncture report was included.Finally,920 patients were included,and in addition to 36normal pancreases,the remaining 884 patients had definite pancreatic lesions.Among the 402 patients with PCLs diagnosed by endoscopy as pancreatic space occupying lesions,49 cases had definite pathological diagnosis(EUS-FNA puncture pathology/surgical pathology).The morphological features of EUS images of these 49PCLs patients were reviewed independently by three EUS physicians.The results showed the highest agreement on the presence or absence of marginal calcification(κ=0.651,p<0.01),there was moderate agreement on the presence of wall nodules,septa,complete cyst wall,solid composition,main pancreatic duct dilation,communication with pancreatic duct,and lymph node enlargement,while there was poor agreement on the shape,number,and presence of thick walls of cystic lesions.In the agreement analysis excluding individual EUS physicians in turn,there was no significant change in kappa values of various morphological characteristics of PCLs.When PCLs morphological characteristics were used for specific classification evaluation,only 21 of the 49 patients were directly evaluated by initial EUS physicians for the type and nature of lesions,and only 12 cases were consistent with the final clinical diagnosis.The diagnostic accuracy rate was 24.5%(12/49).The three EUS physicians who participated in image review lacked confidence in directly reporting endoscopic diagnosis conclusions because they could not reproduce the dynamic images during examination,and the diagnostic accuracy was lower.By combining the results of EUS morphology and EUS-FNA cytopathology,27 cases were consistent with the final clinical diagnosis,and the accuracy rate could be improved to 55.1%(27/49).Conclusion 1.Cytopathologists evaluated the agreement of EUS-FNA cytological diagnosis in all SPLs patients with relatively high,but the consistency of cytological diagnosis and specimen evaluation in the false-negative group was not ideal.The high false-negative rate of initial cytological diagnosis is mainly related to the strict and cautious evaluation criteria of cytopathologists.The inconsistencies of cytological diagnosis and the number of diagnosed cells may be related to the occurrence of false-negative diagnosis.The number of diagnostic cells and blood turbidity index are important factors leading to false negative diagnosis.Therefore,refinement of cytological diagnostic criteria,standardization of specimen quality evaluation and the standardized training of cytological pathologists may improve the agreement of cytological diagnosis and reduce the incidence of cytological false negative diagnosis.2.When EUS physicians evaluated individual morphological features of PCLs,most of the indicators only had moderate IOA,and the accuracy of diagnosis for PCLs solely based on morphological features was also very low.It is urgent to further strengthen the standardized training of EUS physicians,unify the descriptive criteria of EUS morphological characteristics,and reduce the influence of variability among EUS observers as much as possible in combination with the development of various EUS interventional diagnosis and treatment techniques,so as to improve the differential diagnosis efficacy of EUS for PCLs.