An investigation of subjective bias/halo in 360-degree multi-source assessment through differential item function and structural equation model

Multiple Source Feedback (MSF) of performance appraisal is being used in organization for more than three decades. Organizations believe that obtaining the evaluative performance ratings of an employee from multiple sources who interact with the ratee (i.e., the person being assessed) will provide a better picture of that employee's performance.;The purpose of this study is to investigate the subjective bias/halo in a 360-degree assessment. This study compares three statistical methods for detecting halo (subjective bias) in a 360-degree assessment, using polytomous response data (e.g., multiple rating scales). A primary, fundamental research question is, Are DIF (Differential Item Functioning) procedures more sensitive for detecting subjective rating bias (halo) than: (a) computing mean differences between rating source groups (boss, peer, and subordinates), or (b) conducting a multi-sample Confirmatory Factor Analysis (CFA)? The overarching hypothesis is that DIF, due to its focus on item-level residual covariances, will be more effective in uncovering systematic bias for one or more rater sources than using ANOVA or multi-sample CFA.;A second question concerns dealing with subjective bias halo. That is, does removing items exhibiting DIF from the rating process mitigate any apparent halo effect? If successful, it is hoped that the answer to this question will result in a recommended "data cleansing" and associated score-aggregation strategy that can be practically implemented for any 360-degree assessment. An empirical investigation meant to answer this second question will not be undertaken in this study, however. That will be left to future research.