| Frequently researchers base their decisions and interpretations on conclusions drawn from data analyses, but what happens when the data used in the analyses are collected from unreliable instruments or surveys? The instrument may include a particular question that invokes different interpretations based on group membership, thereby placing one of the two groups at an unfair disadvantage. Another challenge occurs when the size of the sample under investigation (i.e., number of respondents or participants) is unavoidably small, adding more uncertainty to parameter estimates. Over the past 50 years, researchers have suggested many different approaches for identifying problematic questions (i.e., items that are biased), but no consensus has been reached as to which method is best. In addition, selecting appropriate methods becomes even more challenging when smaller sample sizes are involved (Lai, Teresi, & Gershon, 2005). This dissertation presents the findings of a study introducing a new method for identifying DIF and potentially biased items. The study explored the use of the Item Information Curve (IIC) as a weighting strategy (i.e., Weighted Area Method -- WAM) to the area between Item Characteristic Curves (ICC) as a way to identify problematic questions. Through thousands of simulations, the performance of WAM was compared to two other commonly used methods for detecting DIF -- the Mantel-Haenszel approach (Mantel & Haenszel, 1959) and the Rudner's Area method (Rudner, 1977). The results show the effects of sample size variations on identifying modeled DIF items, and the opportunity for future Differential Item Functioning (DIF) analyses using WAM. |
