Clustering-Based Cognitive Diagnosis Method And Its Application

In recent years,online assessment technology has been widely used in the National College Entrance Examination(NCEE).There is a tremendous amount of scientific and complete NCEE Data.At the same time,China has formulated some individual education policies,and hope to improve the quality of teaching.However,the study based on the NCEE Data mainly focuses on the control of assessment error,the protection of examinee's privacy,or other aspects.There is a little research about learning outcomes evaluation and individual teaching strategies formulation through the NCEE Date.Now,how to use these NCEE Data to provide guidance for individual teaching has become a research hotspot.Cognitive diagnosis methods provided by Psychologists could diagnose students' Attribute-mastery Patterns(AMP)through test data,and evaluate students' learning outcomes at the micro level.Analyzing mastery proportion for each attribute of students in each school,could evaluate the teaching quality of schools from a microscopic perspective.Based on the evaluated results,we could organize individual teaching better,and improve teaching quality.Test data could be categorized into dichotomous response data,polytomous response data,and mixed response data which consist of dichotomous response data and polytomous response data.Investigations have revealed that available cognitive diagnosis methods for dichotomous response data contain parametric methods and non-parametric methods.Parametric methods with the aid of complicated mathematical model,and non-parametric methods apply mature intelligence algorithms.Comparing with parametric methods,the diagnostic accuracy of non-parametric method is lower.In order to improve the accuracy of diagnosis,we combine the idea of parametric methods and non-parametric methods,proposing a cognitive diagnosis method for dichotomous response data.Considering the weights of cognitive attributes in test items,many cognitive diagnosis methods for dichotomous response data have been expanded.We expand our cognitive diagnostic method for dichotomous response data to apply to mixed response data.At present,a small number of cognitive diagnosis methods have been used to a single test data to diagnose students' attribute-mastery patterns;but few methods have been used to evaluate the school's teaching quality through test data for years.In this paper,the cognitive diagnosis method for mixed response data is applied to the NCEE Data which comes from three high schools in a city for five consecutive years.We analyze the changes in the teaching quality of the three high schools over the past five years from the micro level.The main contributions of this paper are as follows:(1)Combining Item Response Theory(IRT),Q-matrix theory and K-means algorithm,we propose the Q-I-K-means algorithm for dichotomous response data.The performance of Q-I-K-means algorithm is analyzed by a simulation study.For simulated data,we consider four kinds of Attribute Hierarchy Structures(AHS)and four different slips,select three frequently used cognitive diagnosis methods for dichotomous response data;and use Pattern Match Ratio(PMR)and Marginal Match Ratio(MMR)as performance measures.A comparative analysis of PMR and MMR of the four cognitive diagnosis methods under 16 different conditions,we find that the diagnostic accuracy of Q-I-K-means algorithm has higher PMR and MMR,and it is less affected by different AHS and slips.(2)Combining Q-matrix theory and K-means algorithm,we propose Q-K-means algorithm for mixed response data.As the same as Q-I-K-means algorithm,we also use a simulation study to analyze the performance of Q-K-means algorithm.The difference is that only one cognitive diagnosis method for mixed response data is selected as the compared method of Q-K-means algorithm.A comparative analysis of PMR and MMR of the two cognitive diagnosis methods under 16 different conditions,we find that Q-K-means algorithm has higher PMR and MMR,and it is less affected by different attribute hierarchy structures and slips.(3)Applying Q-K-means algorithm to the NCEE Data of mathematics.The NCEE Data we used includes test papers from 2013 to 2017,and response data from 2013 to 2017 respond by candidates from three schools with different teaching quality.We first diagnose attribute-mastery patterns of candidates form different schools in different years,second analyze the master proportion of various mathematics abilities for different schools in different years to evaluate each school's teaching quality every year,and third analyze the changes in the teaching quality of the three high schools over the past five years.The Q-I-K-means and Q-K-means algorithms we proposed would effectively improve PMR and MMR,and reduce the impact of attribute hierarchy structures and slips.Applying Q-K-means to the NCEE Data,we diagnose that candidates master or not for various mathematics abilities and then evaluate the teaching quality of schools.