The Estimation Of Test Cognitive Attributes Without Prior Information

Cognitive diagnostic assessment(CDA)has been widely used in the field of psychological and educational assessment.CDA is based on the cognitive attribute structure(called Q-matrix)of the test,and infers the latent cognitive attribute mastery mode according to the observed response,so as to provide multi-dimensional and finegrained diagnostic information for the subjects.It is evident that the specification of the Q-matrix is the groundwork of CDA.The Q-matrix can be calibrated by domain experts or estimated based on response data.The Q-matrix of expert calibration can directly reflect the domain knowledge structure.Still,it is easy to be influenced by the expert’s subjective cognition,and the calibration cost is high.The Q-matrix estimation based on response data can reduce the subjectivity of Q-matrix calibration and the burden of experts.Therefore,data-driven Q-matrix estimation methods have become a research hotspot in cognitive diagnosis.As a result,researchers have developed many Q-matrix estimation methods based on different research assumptions.According to whether prior information is relied on or not and what prior knowledge is relied on,the existing data-driven Q-matrix estimation methods are divided into four categories,namely:Q-matrix estimation method with known initial Q-matrix,Q-matrix estimation method with known partial q-vector,Qmatrix estimation method with only known attribute numbers and Q-matrix estimation method without any prior information.The Q-matrix estimation method without any prior information estimates the number of latent attributes K and the elements of Q-matrix simultaneously from the response data,which not only saves a lot of manpower and material resources,but also avoids the influence of inaccurate prior information on estimation accuracy,it is has vital practical significance.However,there are still some limitations in the existing Qmatrix estimation methods which don’t depend on prior information:(1)The existing estimation algorithms are all parametric methods,which are complicated in calculation and estimation process and are difficult to apply and promote.(2)In more complex practical application scenarios,such as a large number of attributes(such as K≥5)and high correlation between attributes(such as ρ≥0.8),the estimation accuracy of the existing methods decreases significantly or even fails to be estimated.(3)The estimation of the number of attributes K depends on the true K.Only by setting the interval of K near the truth value,can a higher estimation accuracy be obtained,and the true K is usually unknown in advance.Therefore,it is urgent to develop exploratory Qmatrix estimation methods from a new perspective,which are simpler,more accurate,more adaptable to more complex test situations,and more objective in inferring the number of attributes.Matrix factorization(MF)is a classical method in machine learning,which represents a matrix as the product of two or more low-rank matrices and is commonly used in dimensionality reduction,clustering,pattern recognition,and image analysis.Based on the advantages of MF in dimensionality reduction,low time complexity and strong scalability(adapt to various dimensions).This paper improves the traditional MF methods and makes it suitable for exploratory Q-matrix estimation in CDA.Firstly,based on the popular DINA model,a Boolean matrix factorization(BMF)method is proposed and its performance is verified.Secondly,a sparse non-negative matrix factorization(SNMF)method is proposed to estimate Q-matrix for a more general GDINA model.Additionally,in order to ensure the availability of the estimated Q-matrix in the context of high correlation of attributes,a majority class symmetric undersampling(MCSU)method is proposed based on the resampling idea in machine learning.Accordingly,the following four studies were carried out:Study 1:Boolean matrix factorization(BMF)Q-matrix estimation method under the DINA model.Firstly,the Q-matrix estimation problem under the DINA model is transformed into an approximate BMF problem,and an approximate BMF algorithm is developed,then combined with the BMF algorithm and statistic index Ratio defined according to the residual of BMF to estimate the number of attributes and Q-matrix simultaneously.The simulation results show that:(1)BMF method has higher estimation accuracy under various experimental conditions,and the estimation accuracy and efficiency are better than the existing RBMs method.(2)The BMF method can obtain stable and accurate estimation results under a small sample size(such as N=500).(3)The estimation accuracy of BMF method under high quality test is significantly higher than that of low quality test,but the estimation accuracy of low quality test can be improved by reasonable Q-matrix design or increasing the length of test.Study 2:Sparse nonnegative matrix factorization(SNMF)Q-matrix estimation under the G-DINA model.Due to the limitation of the DINA model,this study mainly explores the Q-matrix estimation under more general G-DINA model.Firstly,the Q-matrix estimation problem under the G-DINA model is transformed into the SNMF problem,and the algorithm is designed.Then,the number of attributes and the Q-matrix are estimated simultaneously by combining the SNMF algorithm and the statistic index Ratio,which is defined by the residual sum of squares of the SNMF.The simulation results show that:(1)SNMF method can obtain satisfactory estimation results under various experimental conditions and is superior to the existing RBMs methods in estimation accuracy and efficiency.(2)The accuracy of attribute number estimation can reach 100%under the fixed Q-matrix design.It can also reach more than 90%under the random Q-matrix design by increasing the test length(such as rising to 30 or 40 according to the size of K).(3)The accuracy of Q-matrix estimation is affected by sample size,Q-matrix design,item quality,number of attributes and test length,etc.However,when the sample size is increased to 4000,the influence of other factors can be almost ignored,the estimation results are stable and the accuracy is over 98%.Study 3:Q-matrix estimation method in the context of high correlation between attributes.This study mainly deals with the problem of the Q-matrix estimation method in the case of high attribute correlation.Firstly,the SNMF method proposed in study 2 was applied to the test situation with correlations between attributes,and the influence of the different correlation levels(low,medium,and high)between attributes on the estimation of Q-matrix are discussed.Then,aiming at the difficulty of estimating the Q-matrix in the context of high correlation between attributes,a new method named majority class symmetric undersampling(MCSU)is proposed based on the idea of resampling in machine learning.The simulation results show that:(1)MCSU can solve the problem of Q-matrix estimation in the context of high attribute correlation.If the undersampling ratio r ∈[0.25,ρ/2],the recovery of both the attribute number and the Q-matrix elements can be improved,and even can reach the same level as when the attributes are independent.(2)The improvement of estimation accuracy by MCSU method is affected by factors such as correlation coefficient,item quality,number of attributes,etc.The increase is more significant under the conditions of the larger correlation coefficient,lower item quality,and more attributes.Study 4:Performance verification of matrix factorization method under real data sets.The feasibility and performance of BMF and SNMF approaches were verified under real data sets.The results on real data sets show that the two new methods maintain the excellent performance of simulated data.Furthermore,compared with the existing RBMs method,the proposed methods have higher estimation accuracy,efficiency,and data fitting.In conclusion,the two Q-matrix estimation methods proposed in this paper have higher estimation accuracy under various experimental conditions and are better than the existing RBMs.Furthermore,the proposed technique of MCSU can solve the problem of Q-matrix estimation in the context of high attribute correlation.The new method enriches the research of Q-matrix estimation in cognitive diagnosis,especially the Q-matrix estimation without prior information.It expands the scope and possibility of the application of CDA in practice.