An improved measure of reading skill: The Cognitive Structure Test

This study compared the construct validity and the predictive validity of a new test, called the Cognitive Structure Test, to multiple-choice tests of reading skill, namely the Armed Forces Vocational Aptitude Battery Paragraph Comprehension Test (ASVABpc) and the Scholastic Aptitude Test (SAT). To test the hypothesis that the Cognitive Structure Test is a better test of reading skill, 347 Air Force recruits read 16 technical and scientific passages (8 from the ASVABpc and 8 from the SAT) and took both multiple-choice and Cognitive Structure tests on the passages. For each passage the recruits' Cognitive Structure Test responses were compared to the responses of two experts who also read the passage and took the Cognitive Structure Tests (a total of 32 experts). These experts also took the multiple-choice tests.;Results indicated that the two tests measured very similar constructs and had similar reliability (Chronbach's Alpha) ratings (Cognitive Structure Test,.63; Multiple-choice test,.68).;Results showed that the Cognitive Structure Test was superior to the multiple-choice tests in predicting final Technical School grade point averages for the recruits, and equivalent to the multiple-choice tests in predicting the ASVAB vocational and general science tests, and general knowledge tests. For the ASVAB passages, the Cognitive Structure Test predicted the final Techschool scores, but the multiple-choice test did not. Both tests predicted domain specific knowledge and general ability.;Correlations between the Cognitive Structure responses of the two experts from each passage and the scores of the experts on the multiple-choice questions were reported. The mean correlation between Expert 1 and Expert 2's cognitive structure responses for the ASVABpc passages was.75, and for the SAT passages was.62. The experts correctly answered 93.8% of the ASVABpc passage multiple-choice questions, but only 82.9% of the SAT passage multiple-choice questions.;Graphical representations of the structures elicited by the Cognitive Structure Test were presented for two passages using Addtree (Sattath & Tversky, 1977) and Extree (Corter & Tversky, 1986).;Suggestions for improving the Cognitive Structure Test were made, along with a review of the unresolved issues.