A framework for science achievement and its link to test items

This dissertation study attempted to develop a working framework of science achievement and link it to the test items selected from the achievement test in the Third International Mathematics and Science Study (TIMSS). It began by reviewing what we know about the notion of science achievement. Then, a theoretical framework for science achievement was developed that (1) defined science achievement as declarative, procedural, schematic, and strategic knowledge; (2) used an examinee-test interaction perspective to explain how students bring and apply their knowledge to answer test items; and (3) linked types of knowledge to test items. Subsequently, the dissertation explored to what extent the underlying patterns of test scores and the cognitive activities displayed when people responded to the items corresponded to the link between items and these four types of knowledge.; The study used various methods. First of all, a logical analysis identified affordances and constraints for the use of knowledge in the test items and thereby linked items to knowledge types. Then, covariance analysis statistically modeled students' item scores and ascertained whether competing knowledge-factor models accounted for the underlying relationship of the scores. Protocol analysis collected participants' verbal reports while responding to test items. By tracing participants' cognitive activities, the dissertation examined whether the test items that were claimed to tap into different types of knowledge evoked participants' use of different science knowledge.; Analysis of the TIMSS items, students' scores, and participants' verbalizations revealed that TIMSS items could be identified according to their affordances and constraints, and thus linked to the four types of knowledge. Most multiple-choice and free-response items tended to be linked to declarative and schematic knowledge, whereas the performance tasks tapped into students' procedural knowledge. The TIMSS multiple-choice and free-response items did not test students' strategic knowledge at all. The covariance analysis was conducted with test items from Booklet 8 and Booklet 4 of TIMSS. The knowledge factor models fit well with the Booklet 8 items, in which the three hypothesized factors of declarative, procedural, and schematic knowledge were extracted from the selected science items. However, factor analysis with Booklet 4 items only partly supported the knowledge model so that the declarative- and procedural-knowledge factors emerged from the item scores, but the schematic-knowledge factor was hidden by those two factors. Finally, the protocol analysis provided cognitive evidence bearing on the distinctions between the knowledge items. The patterns reflected by participants' use of knowledge in solving the test items were consistent with the item pre-classifications from the logical and covariance analyses. Overall, the logical, covariance, and protocol analyses converged to generally support the claims about the distinctions between the three types of science knowledge and justify their links to test items. (Abstract shortened by UMI.)...