Generalizability theory: Measuring the Dependability of Selected Methods for Scoring Classroom Assessments

This study compares the dependability of four classroom assessment scoring methods. Generalizability theory (G) and alternative decision (D) are used to measure the results of students' classroom assessment scores and compare the results of the four scoring methods on variability of rater by person variance and the level of G and D coefficients within each study. This research study builds on a similar study by Marzano (2002) and contributes to a body of knowledge by filling in the gaps about how classroom teachers score assessments.;This generalizability study reports the findings from a two-part study. In the first part of the study, an unconstrained point method of scoring short-answer social studies assessments is compared to a generic rubric method of scoring. In the second part of the study, a constrained point method of scoring short-answer social studies tests is compared to a topic-specific rubric smethod of scoring.;This study suggests that, generalizability theory (G) is used to estimate the amount of variance in scores from one rater to another within each scoring method, while comparing the variability across methods and the two parts of the study. In this study, generalizability analysis results help the educational community understand the relative importance of various sources of measurement error and also assist in choosing and redesigning efficient measurement procedures for scoring classroom assessment meant to minimize error and maximize reliability.