Development And Application Of IRT Model Incorporating Response Time In Personality Forced-Choice Questionnaires

Personality assessment is widely used in many real-life applications.Traditionally,the most popular assessment tool is Likert scale,where participants are asked to rate a statement using several given categories(e.g.,ranging from ’’strongly disagree’’ to’’strongly agree’’ or from "never’’ to ’’always,’’ etc.).However,such a format can lead to various response biases,for example,response styles,social desirability,and halo effect,etc.In addition,when Likert scales are used in some high-stakes scenarios(such as personnel selection),interviewees are easy to distort their answers to obtain a good performance.These false scores seriously violate the principle of authenticity of the test,and greatly reduce the validity of personality test.Forced-choice questionnaires(FCQs)were developed to address these issues.In FCQs,the respondents are presented a certain number of blocks,each containing two or more items,and these items usually measure personality traits in different dimensions.respondents have to make comparative judgments,choosing between several items according to the extent to which the items describe their preferences or behavior.Moreover,items within each block are often matched in terms of their level of social desirability or perceived relevance,which reduces the potential for faking.Numerous studies have provided empirical evidence that FCQs can control response biases better and have greater predictive validity than Likert scales.However,FCQs have been heavily criticized because their traditional scoring produced ipsative data.For an individual,the score of one trait relies on the scores of other traits measured.Ipsative data are inappropriate for comparisons between individuals,and impair the reliability and validity of the test.To overcome problems associated with ipsative data,some recent studies have proposed new scoring procedures within the framework of item response theory(IRT),These methods can handle FCQs’ data very well and produce normative scores that could be used for comparisons between individuals,around which several studies and applications have emerged.Moreover,given the prevalence of computer-based tests,many institutions are now using computer-based tools to conduct surveys for data collection,and the precise item-level response times(RTs)can be recorded accurately and easily.As a metric generated in conjunction with the responses,RTs are a potentially valuable source of information concerning the test-taker’s behavior and the item characteristics.This drives us to use this information to further explore the response process,and substantial progress has been made in the field of cognitive testing,where a large number of different models for responses and RTs have been proposed.Given this situation,this study proposed a new IRT model that incorporates response times in FCQs to improve personality assessment.Simulation studies show that the proposed model can effectively improve the estimation accuracy of personality traits with the ancillary information contained in RTs.Also,an application on real data reveals that the proposed model estimates similar but still different parameter values compared to the conventional Thurstonian IRT model and these differences can be explained by the RTs information.At the same time,the model also can help us to explore the individual’ latent response process to the items in FCQs,then we can obtain more abundant diagnostic results.