Enhancing structural knowledge assessment: A new design to measure conceptual relatedness

The structural assessment paradigm asserts that knowledge organization is a determinant of domain expertise and that a person's cognitive structure can be inferred by eliciting judgments about conceptual relationships in an expert-defined domain. For decades, concept pairs relatedness ratings have been used in research to assess structural knowledge, predicting conventional measures of proficiency in many domains. However, the methodology has several drawbacks that limit its application in typical learning environments.; Experiment I demonstrates a method that improves the efficiency of the all-pairs relatedness test by selecting an expert-determined subset of pairs for an abbreviated test. Subsets were extracted from elicited all-pairs data in two domains, first with laboratory data and then with college-level academic course data. In both domains it was shown that the extracted subset data retained the structural properties attributed to all pairs and correlated with conventional exam scores as well as did the all-pairs data.; Experiment II extends the subset ratings test with additional expert-determined terms and pairs, increasing domain coverage beyond the practical limits of an all-pairs test. Study 1 describes an enhanced global relatedness test for administration at the beginning or end of a course of study. In Study 2 the global test was subdivided and new pairs were added to create four brief, topically focused tests for use while learning progresses. Both the global and local enhanced relatedness tests correlated with the course exams as well as did the all-pairs and extracted subset tests, with no significant difference between the global and local enhanced tests.; Experiment III addresses the face validity of the relatedness ratings test. The elicitation and scoring methods are unfamiliar to students, and individual item responses are difficult to interpret along with similar conventional test items. To address these concerns, multiple-choice relatedness tests were developed from the local enhanced test pairs. The choice relatedness test scores correlated with course exams as well as did the relatedness ratings tests.; Applications of the enhanced relatedness test include timely identification of misconceptions to facilitate cognitive restructuring. Implications are discussed for advancing theories of knowledge acquisition and organization in formal and less well defined domains.