Research On The Improvement Of RSM And The Polytomously-scored AHM

The true score scale in CTT or the latent trait scale in IRT provides specific information about students'location on a continuum. However, it fails to provide specific information about students'cognitive strengths and weaknesses. Diagnostic assessments enable researchers and educators to make inferences about cognitive processes and knowledge which students use when solving test items. The value of diagnostic assessment lies in its ability to reveal each student's specific cognitive strengths and weaknesses and further help design effective interventions for individual students. Tatsuoka's groundbreaking works on the rule-space model (RSM), published over a period of more than twenty-five years, serves as one of the first psychometrically-based methods for diagnosing cognitive attributes. Tatsuoka believed that assessments could and should yield specific information about examinees'cognitive skills and guide instructional decisions. The attribute hierarchy method evolved from the rule-space approach, is used to classify examinees'test item responses into a set of structured attribute patterns associated with different components from a hierarchical cognitive model of task performance.In this study, some flaws of RSM are discussed and improved. The first part of RSM is Q matrix theory, the second part is pattern recognition. There are some errors about the Q matrix theory proposed by Tatsuoka. Some remedial measures and simpler methods of generating ideal item-response patterns are proposed. In addition, The method to the classification in RSM is complicated. Thus several methods for the classification of cognitive competencies were proposed. Monte Carlo simulation was used to compare the following methods for classification: approaches which was used in RSM, some methods represent distance including Kullback-Liebler entropy (KL), Chi-Square distance (KF), Square Root (SQR), Logarithm of Likelihood(LL)and cosine (COS) which represents similarity between two patterns. Simulation results showed LL is the best for classification; KL, KF, SQR, COS have the same result as that in RSM, but the computation of these methods is simpler.In this paper, an extension of the AHM for polytomously-scored items (hereafter referred to as the Poly-AHM) is proposed, and classification method - logarithm of likelihood ratio (LL) which perform best in RSM for the classification is used in Poly-AHM. In addition, the method that expected response patterns based on polytomous model derived from the reduced incidence matrix Qr is introduced. A series of new classification methods (S1, S2, S11, S21) are proposed which is based on establishing a series of indices of the similarity between the expected response pattern and the observed response pattern. Simulated response vectors were used to evaluate the performance of the classification methods (LL, S1, S2, S11, S21, Method A and Method B) based on Graded Response Model for the AHM. The performances of these classification methods were evaluated at the attribute pattern level and the individual attribute level, respectively. At both the attribute pattern and the individual attribute level, the results of simulation showed that LL and Method A performed best among all the classification methods, S2 and S21 outperformed S1, S11 and Method B. These methods performed considerably higher at the individual attribute level than at the attribute pattern level. Moreover, the trends of the classification rate varied with the slip are consistent at all kinds of attribute hierarchies, while S2, S11, S21 performed inconstantly at the attribute hierarchies of Linear and Convergent. In addition, Poly-AHM is better than the polytomous extension of the Fusion Model proposed by Bolt et al. in terms of the classification accuracy and simplicity.