Reliability of a patient classification system: An application of generalizability theory

The primary purpose of this study was to identify the appropriate methods for determining the reliability of patient classification systems. The methodological problems in designing a generalizability study to estimate the components of variance important for estimating PCS reliability were examined. Several generalizability study models were designed that incorporated the measurement facets relevant to patient classification. A demonstration of one of the proposed designs was conducted.;A single facet model was designed to examine the potential effect of different raters on measurement. The model was designed so that the same pair of nurse raters, a randomly selected nurse and the investigator, rated all measurement subjects (patients). The patient sample consisted of 30 patients. One patient was dropped from the analysis due to incomplete data.;Variance components were estimated for each of the two main effects (nurse raters and patients) and the residual error. A coefficient of generalizability, analogous to the reliability coefficient, was computed from the variance components estimates. The generalizability coefficient for the total PCS score was 0.034. With one test item deleted, the generalizability coefficient increased to 0.650. The generalizability coefficient for individual items ranged from 0.053 to 0.961.;Based on the findings of this study, recommendations for improving the reliability of the patient classification system were offered. Items with poor reliability were identified for revision or deletion. An ongoing training program to improve the rating skills of the nurses was also suggested to increase interrater reliability. Although the findings of the study were not generalizable to other patient classification systems, the application of generalizability theory to the question of interrater reliability was shown to be superior to other methods of determining PCS reliability.;Unit directors on 10 patient care units at a medical center located in a midwestern city were queried about the conditions that are present in the patient care delivery process that may influence patient classification. Three categories of measurement conditions, called facets, within the patient care delivery system were (a) nurse rater facets, (b) nursing unit facets, and (c) time facets.