The Application Of Kappa In Assessing Agreement

Objective To explore an approach which can be used to estimate kappa, when covariates are involved in measuring agreement. Discussing some constraint in the application of kappa, and exploring their reason in order to propose some feasible methods.Methods Generalized estimating equations would be employed to estimate the marginal probabilities and kappa, and an appropriate explanation of those parameters would be given, in order to provide a guideline in medical research. Exploring the characteristics of kappa and relation between kappa and ρ by Simula- tion. Some examples used to illustrate the reason of those constraints of kappa, and to discuss how to use some related solution.Results Generalized estimating equations is an approach which can be used to identify covariates predictive of kappa; it is not only used in dichotomous data, but also used when the characteristic measured on an ordinal scale. The distribution of kappa is symmetry in most cases, but it appears to skew when the marginal probabilities of raters close to 0 or 1. So the approximate approach is not always applicable, which to take hypothesis or calculate the credibility interval of kappa, and some correction for results or other calculating methods is necessary. The relation between kappa, ICC and ρ is complicated, so researcher should pay more attention to avoid some misapplications. Kappa is influence by the original distribution of subjects and bias of raters, so multiple indexes should be calculated in measuring agreement.Conclusion Generalized estimating equations approach can be used to identify covariates predictive of kappa. Measuring agreement is comprehensive, and those constraints of kappa can be avoided.