Research On The Attribute Specification And Model Development In Cognitive Diagnostic Assessment

Cognitive diagnostic assessment(CDA) is a new style of test, it can infer the knowledge-state of examinees based on their responses to items which have diagnostic capability. Relative to the traditional test style, CDA could offer more useful information for studying, instructing and evaluating, and it has already attract much attention of the whole society. Our national outline for Medium- and Long-Term Educational Reform and Development(2010-2020) has explicitly mentioned that “attach high importance to the cultivation of students’ abilities and their strong suits, and pay attention to students with difference characteristics and individual differences, develop the advantage and potential of each student, ……, reform education quality evaluation and talent evaluation system”. To teach in accordance with students’ aptitudes, different characteristics and individual differences of students should be known in advance which the advantage of CDA is.To conduct cognitive diagnostic assessment, there are some key issues should be addressed first, and this dissertation mainly focuses on three of them. The first one is the definition of the item-attribute-vector(or the specification of the item-attributevector), and it contains two parts: validating the attribute-vector of prespecified items and defining the attribute-vector of undefined items or new items. Generally speaking, these two parts are commonly defined by multi-disciplinary specialist and measurement experts based on their knowledge and experience. The second issue is the influence of attribute granularity to CDA. The third one is the development and usage of cognitive diagnostic model. For the first problem, the main method is the definition by experts, and this might be subject to be affected by the knowledge and personal experience, so the phenomenon that disagreements among different experts and controversy always appears. Up to now, the famous “fraction-subtraction”(Tatsuoka, 1983) data, its expertspecified Q matrix is still controversial, and it is more than twenty years from the time in which the data was collected. De Carlo(2011, 2012) mentioned that the attribute defining in CDA was a very complex work. There are correlation between the first problem and the rest two problems, and the second and the third problem will not get a proper settlement unless the first one has already get addressed appropriately. The definition of item-attribute-vector will concern the choice of the attribute granularity, and the choice of the cognitive diagnosis model is based on the definition of the attributes and their relationships, so choosing an inappropriate model to use is probable in real applications.It is exactly based on the consideration of the problems in CDA, and combining the present situation of overseas and domestic research, this dissertation conducts four studies.The first study contains three parts. Part 1: Liu, Xu and Ying proposed a Q matrix inference method based on response data, and in simulation studies, their method has a high successful recovery ratio, and it is an objective approach for estimating the Q matrix. The usage of the method might be limited because of much assumptions, so consider relaxing some constraints and jointly estimating the Q matrix, item parameters, further, consider relaxing the assumption that examinees in the population are known in advance, and jointly estimating the Q matrix, item parameters and the distribution of the population.Part 2: There should be a good “initial Q matrix” for the Part 1, and a good “initial Q matrix” means less misspecifications in it, yet this is not always hold in real applications, this part can online estimate the Q matrix and item parameters based on those items whose attribute-vector has been correctly prespecified, and further, online estimate the Q matrix, item parameters and the distribution of the population. Relative to the Part 1, the advantage of the Part 2 is that it can estimate the Q matrix based on a more general Q matrix(means that the Q matrix only contains several correctly prespecified items), not a good Q matrix. This is very beneficial for the constructing of item bank in real applications, and the calibration of the new items can be fulfilled based on the prespecified items.Part 3: Currently, it is assumed that the attribute-number is always correctly predetermined in most researches on CDA, but the determination of the attributenumber is not an easy work in real applications, such as the famous “fraction subtraction data”, it was analyzed in according to 8 attributes and 5 attributes by some researhcers. This study consider the performance of the Q-matrix estimaton algorithm under the condition of lacking of a required attribute or adding a redundantattribute.Regardless of the quality of the prespecified Q matrix, the Part 1 or Part 2 of the first study can be quite competent to infer the Q matrix. On the one hand, the parameters and the attribute-vector of the “new items” can be calibrated. On the other hand, it also can validate the attribute-vector of the prespecified items. The drawbacks of the study 1 is that it needs an intensive computation especially when the attribute-number or the item-number is bigger, and the algorithm may not provide the output immediately. So, consider constructing a new statistic which needs a less intensive computation, and also has a high successful recovery ratio.Motivated by the 2 statistic in item response theory, a similar statistic names as 2 was constructed within the framework of cognitive diagnosis as well as the algorithms. The algorithms based on the 2 statistic can not only jointly estimate the Q matrix and item parameter, but also can online estimate the Q matrix and item parameter, and most importantly, the advantage of the 2 statistic is that it doesn’t depend the distribution of the sample.Up to now, there is still lack of the researches which concern the effects of attribute granularity to CDA. So, study 3 focuses that the effects of attribute granularity and the correlation among attributes to the classification of examinees.Generally, it is very difficult for researchers to determine the relationships among attributes, and most of the cognitive diagnostic models are constructed based on some kind of attribute-relationship. The model and the data would not fit well when the chosen model doesn’t match the actual attribute-relationship, and this might lead to reduce the pattern classification correct ratio and decreased the value of the information provided by CDA. Consider adding a parameter which can depict the attributerelationship for every item, and this parameter can model the effect which is produced by the relationship among attributes. This newly constructed model will have a better adaptability, and it can not only address the conjunctive relationship among attributes, but also manage the fully and partial compensation relationship. Most importantly, it doesn’t need to determine the relationships in advance, and the information about the attribute-relationship can be provided by the extra parameter, as well as the size of the relationship. Relative to the existing model, the proposed model has better performance in different kinds of test data.