| The Statistics Concept Inventory (SCI) is one of several concept inventories currently being developed in a variety of engineering disciplines following the success of the Force Concept Inventory (FCI). The direction of the current reform movement in statistics education (as well as other science, engineering, and mathematics fields) is toward an emphasis on conceptual learning instead of focusing on procedural and computational skills. These new curricular goals have given rise to new assessment needs. The SCI is a multiple choice instrument modeled after the FCI which aims to assess conceptual understanding of fundamental statistics concepts. Development of the SCI began in 2002. An overview of the development process is presented here along with baseline performance data from a variety of university level statistics courses. SCI data is analyzed from a classical test theory perspective and from an item response theory (IRT) perspective using the two parameter logistic model and the nominal response model.; Posttest SCI results have been consistently low, between 40% and 50% correct; pretest to posttest gains have been minimal. These outcomes are consistent with concept inventory findings in other disciplines. As part of the ongoing development process, individual item analysis has been conducted including item discrimination, distribution of answers, and item correlation with the total score. Comments from student focus groups have also been used during the revision process. These detailed findings are presented as an annotated version of the SCI. Potential areas of confusion or possible misconceptions can be identified.; A clearer picture of student understanding emerges when the item analyses are combined with analyses obtained using IRT methods. In particular, the nominal response model appears to be able to shed light on persistent misconceptions versus those that seem to diminish with instruction. Additionally, IRT methods can be utilized during the revision process to compare question versions, help make decisions which increase reliability, and make the revision process more efficient. Item characteristic curves for each question and for each response are presented. Results indicate that these methods should be very useful for revising and interpreting concept inventories, as well as having pedagogical implications. |
