Impact of matched samples equating methods on equating accuracy and the adequacy of equating assumptions

When test forms are administered to examinee groups that differ in proficiency, equating procedures are used to disentangle group differences from form differences. This dissertation investigates the extent to which equating results are population invariant, the impact of group differences on equating results, the impact of group differences on the degree to which statistical equating assumptions hold, whether matching techniques provide more accurate equating results, and whether matching techniques reduce the extent to which statistical equating assumptions are violated.;Data from one administration of four mixed-format Advanced Placement (AP) Exams were used to create pseudo old and new forms sharing common items. Population invariance analyses were conducted based on levels of examinee parental education using a single group (SG) equating design. Old and new form groups with common item effect sizes (ESs) ranging from 0 to 0.75 were created by sampling examinees based on their level of parental education. Equating was conducted for four common item nonequivalent group (CINEG) design equating methods: frequency estimation, chained equipercentile, item response theory (IRT) true score, and IRT observed score. Groups with ESs greater than zero were matched using matching techniques including exact matching on parental education level and propensity score matching including other background variables. The accuracy of equating results was evaluated by comparing differences between comparison (ES>0) and criterion equating (ES=0) relationships using the root expected mean squared difference statistic, classification consistency for AP grades, and standard errors of equating. The accuracy of equating results and the adequacy of statistical equating assumptions was compared for unmatched and matched samples.;There was relatively little population dependence of equating results, despite large subgroup performance differences. As ES increased, CINEG equating results tended to become less accurate and less consistent. Large differences between criterion and comparison equating relationships appeared to be caused by violations of equating assumptions. As group differences increased, the degree to which frequency estimation and chained equipercentile statistical assumptions held decreased. All exams showed some evidence of multidimensionality. The matching methods that included parental education appeared to improve equating accuracy and the degree to which equating assumptions held for very large ESs.