Development And Application Of Item Response Tree Model For Aberrant Behavior Under IRT Framework

In standardized tests,for various reasons(e.g.,lack of time,lack of motivation,etc.),test takers may exhibit aberrant response behaviors other than normal responses:rapid-guessing behavior,cheating behavior,missing response behavior,etc.These aberrant behaviors can lead to inaccurate estimation of test taker ability/trait and item parameters,which can undermine the validity and fairness of the test,so it becomes important to find out how to make accurate ability estimation for test takers with aberrant behaviors.To address this issue,this study proposes a mixture IRTree model(MIM-NAAR model)that uses a tree modeling framework combined with an item response theory model to simultaneously consider the effects of three types of aberrant behaviors-rapid guessing,cheating,and missing responses-on test-taker ability estimates and to obtain more accurate estimates of test-taker ability parameters.An outstanding advantage of this model over traditional IRT models is that it allows for more classification of test-taker behavior at both the item and test-taker levels,and it considers and models guessing and cheating separately and simultaneously for the first time.In order to verify the rationality of this newly constructed MIM-NAAR model and its performance,this paper use two batches of real data for the study and compared it with the traditional two-parameter logistic model(2PLM)and three-parameter logistic model(3PLM).Then,a simulation study was conducted based on the parameters estimated from the real data to verify the accuracy and superiority of the new model parameter estimation for different number of test takers,number of items and response missing rate.The results of the study showed that.(1)In the model fit comparison of the two empirical studies,the MIM-NAAR model developed in this study can better fit the real data and the results of parameter estimation are more reasonable.(2)The MIM-NAAR model has good accuracy of parameter estimation under various experimental conditions and outperforms the traditional model.The model’s estimation of item parameters is more accurate as the number of test takers increases,and the model’s estimation of test taker parameters is more accurate as the number of test questions increases.(3)Combining the empirical analysis with the simulation study,it can be concluded that missing responses and rapid guessing behavior tend to lead the traditional model to underestimate the ability of the test takers,while cheating behavior leads the traditional model to overestimate the ability of the test takers.This effect is minimal for the new model,suggesting that the new model is better able to avoid the negative effects of aberrant responses on parameter estimation.