Development And Application Of Item Response Tree Model Based On Answer Changing Behaviors

In high-stakes tests,test takers usually make full use of their time to achieve higher scores.Thus,after making initial responses to items,if only they have ample time,they would return to the items again,check the initial answer and change or maintain it according to their own judgement.The item response tree model can present the implicit intrinsic cognitive process of the answer change behavior clearly,which is more conducive to our understanding of the answer change behavior and further extends the application of the item response tree model.In this paper,a new item response tree model,a statical answer changing IRTree model(the SAC Tree Model for short)based on the existing item response tree model is innovatively constructed by modeling two inner cognitive reactions of revisiting and changing.To verify the feasibility of the SAC Tree Model and its application in real test-taking situation,two simulation studies and one empirical research are conducted in this paper.(1)Simulation study Ⅰ is a two-factor experimental design,where one factor is the number of subjects(three levels,500,1000,and 1500,respectively)and the other is the number of items(two levels,20 and 40 items,respectively).The MCMC method is used to estimate the parameters by using Stan to verify the feasibility and accuracy of the SAC Tree Model.(2)Simulation study Ⅱ further compares the parameter recovery of the proposed SAC Tree Model with the IRTree model of Jeon et al.,and further verify whether the SAC Tree Model has better recovery of parameters estimation than traditional IRTree model of Jeon et al.(3)To further prove the rationality and superiority of the SAC Tree Model in the empirical research,the fitness of the SAC Tree Model and IRTree model of Jeon et al.to the actual response data is compared,as well as the consistency of parameters estimation of the same subjects’ latent and the outliers of the two models.The results show that:(1)The accuracy of the parameter estimation of the SAC Tree Model proposed in this paper is reasonable,and the model information fits well,indicating that the model is feasible.The more the number of test items,the higher the accuracy of the new model for estimating the parameters of subjects.The more the number of subjects,the higher the accuracy of the new model for estimating the parameters of the items.(2)Not only the SAC Tree Model has better recovery for parameters estimation,but it is also more accurate than IRTree model of Jeon et al.,which indicating that the new model has obvious advantages.(3)In the empirical research,the convergence index of parameter estimates for all items of the SAC Tree Model is less than 1.1,indicating that the MCMC algorithm has better convergence of parameter estimates.Meanwhile,the SAC Tree Model fit real data better than IRTree model of Jeon et al.and the latter has outliers,which further proves the advantages of the SAC Tree Model.In summary,this study proposes a new item response tree model based on modified answer behavior,which further extends the application of item response tree model in the study of modified answer behavior.Compare with IRTree model of Jeon et al.,the outstanding advantage of the proposed model is that the proposed new model could give further insight into the cognitive processes and detect more answer change behaviors with higher recovery of parameter estimation.The results of both the simulation studies and empirical research validate the rationality of the proposed tree model.