| Item pool quality has been regarded as one important factor to help realize enhanced measurement quality for the computerized adaptive test (CAT) (e.g., Flaugher, 2000; Jensema, 1977; McBride & Wise, 1976; Reckase, 1976; 2003; van der Linden, Ariel, & Veldkamp, 2006; Veldkamp & van der Linden, 2000; Xing & Hambleton, 2004). However, studies are rare in how to identify the desired features of an item pool for the computerized adaptive test (CAT). Unlike the problem of item pool assembly in which an item pool is assembled from an available master pool according to the desired specifications, no actual items are available yet in the problem of item pool design (van der Linden, Ariel, & Veldkamp, 2006). Since there is no actual item available when designing an item pool, designing an item pool that is optimal intuitively becomes a desired goal. This study is focused on designing an optimal item pool for a CAT using the weighted deviations model (WDM; Stocking & Swanson, 1993) item selection procedure. Drawing on Reckase (2003) and Gu (2007), this study has extended the binand-union method proposed by Reckase (2003) to a CAT with a large set of complex non-statistical constraints. The method used to generate optimal item features is a combination of methods based on McBride & Weiss (1976) and Gu (2007) for statistical features and a sampling method based on test specifications for non-statistical features. The end-product is an item blueprint describing items' statistical and non-statistical attributes, item number distribution, and optimal item pool size. A large-scale operational CAT program served as the CAT template in this study. Three key factors considered to potentially impact optimal item pool features were manipulated including item generation method, expected amount of item information change, and b-bin width. Optimal item pool performance was evaluated and compared with that of an operational item pool in light of a series of criteria including measurement accuracy and precision, item pool utilization, test security, constraint violation, and classification accuracy. A demonstrative example on how to use identified optimal item pool features for item pool assembly was provided. How to apply optimal item pool features to item pool management, operational item pool assembly, and item writing was also discussed. |
