| Evaluation study of diagnostic test plays important roles in providing evidence for its application on clinic, improving medical service, preventing medical source waste. Among evaluations of diagnostic test, evaluation of accuracy is most important. ROC curve is believed to be the best index in evaluation of diagnostic test because it is not affected by disease incidence and cut-off point and is an integrated index with sensitivity and specificity. Furthermore, ROC curve makes it easy to compare accuracies of two or more diagnostic tests.The area under ROC curve, namely Receiver Operating Characteristic curve, can be estimated by either parametric or nonparametric methods. To nonparametric method, the area is usually estimated according to Wilcoxon Mann-Whitney statistic, which is the statistic of rank sum test of comparing the test values of patients and non-patients. Fitting binormal distribution model is the most used one in parametric methods for area estimation. When accuracies of two diagnostic tests are compared by areas, to paired design, the test values of two diagnostic tests are usually correlated. So the statistic value of significance test is correlated to correlation coefficient of two areas or their co variance. This process is very difficult for compute and could often be accomplished by statistic software packages such as ROCKIT, SAS, SPSS.In this study, the diagnostic test of lung cancer was developed recently by a company. It could be used at a time to test three tumor markers, namely Cyfra21-1, CEA, NSE(the cut-off points of which clinically are 3.3ng/ml,5ng/ml, 12.5 ng/ml respectively ). After analysis, it was found that when the sample exclude other tumors except for lung cancer, sensitivities of three tumor markers in the diagnostic test were 54.60%, 60.12%, 44.17% respectively, and specificities of them were 95.49%, 98.01%, 99.82% respectively, which indicated that their sensitivities were low, while their specificities were high. When the sample included other tumors, sensitivities had no change, and specificities of them were 76.30% , 84.13%, 88.61 % respectively,which indicated that their specificities became lower. The analysis of the ROC curves showed:l.when the samples exclude other tumors and include them, the areas under ROC curves of three tumor markers, namely Cyfra21-1,CEA,NSE, were 0.81,0.86,0.70 and 0.70,0.78,0.63 respectively, by using nonparametric estimation method. However, by using parametric estimation method, their areas were 0.81, 0.86, 0.69 and 0.70, 0.77, 0.63 respectively. Comparing these areas with 0.5 which serves as a reference of a test with no diagnostic value, the differences were all significant(P<0.05). These results indicated that Cyfra21-1 and CEA markers had moderate accuracy to distinguish people of lung cancer from others, while NSE had low accuracy.2. The areas estimated by nonparametric and parametric methods were almost equal.3.when the samples excluded and included other tumors, the accuracies of the former were higher than those of the latter on the whole. So if the diagnostic test is used to screen persons with lung cancer from general people, the result of the latter could be used as a reference. If the diagnostic test is used to diagnose people to find out whether they get lung cancer or not clinically, the result of the former could be used as a reference when other tumors were excluded. According to the results of all evaluation indexes, the diagnostic test about lung cancer had low sensitivity and high specificity, and it had moderate accuracy on the whole.This study demonstrated the methods of estimating the area under ROC curve and their applications and the differences between parametric and nonparametric methods. The nonparametric method could be used to estimate the area in evaluating almost all diagnostic tests because it had not any use limitation. But the area obtained was usually smaller than the true area. Specially when there were many same testing values (for example, the testing values were the ranked ordinal data), that problem would become more obvious. While the parametric method of the fitting binormal distribution model could be used to estimate the area under ROC curves more accurately than nonparametric method. But in a few conditions, ROC curve could be located under chance line, or when the distribution of data was far from the conditionrequired, the area value obtained may be quite different from true value. When the sample size was large and the same testing values were few, the areas obtained by parametric and nonparametric methods were almost same, such as the example in this study. In practice, either of the two methods would be selected according to different condition.In this study, the comparison of the new diagnostic test (new test) about PCA and the other one (control test) , which is used usually at present clinically, was performed. And the result shows: to Free-PSA marker, the areas under ROC curves of the new test and the control test were 0.91, 0.92 respectively, and to Total-PSA marker, the corresponding areas were 0.89, 0.90 respectively. These results showed that two diagnostic tests had high accuracy. The difference between them was not significant (P>0.05). The results about equivalence test were that they had the equal areas (P<0.05), and then indicated that two tests had the equal accuracy to diagnose PCA, and also provided important evidence for the application of the new test clinically.With the development and improvement of statistical methods, the method of evaluating diagnostic test is advancing fast. However there are still some problems about them, for example, statistics test of goodness of fit about fitting binormal distribution model, ROC curve analysis in the present of verification bias, regression analysis about diagnostic test, Meta analysis of diagnostic test, evaluation of diagnostic test when the test values were the ranked data, and so on. The further research is important and valuable for these problems existing.
|
PDF Full Text Request