| Measuring performance in the workplace is an endeavor that has been the central focus of many applied researchers and practitioners. Due to the limited information that objective data provides to decision makers, subjective data are often used to supplement performance ratings. Unfortunately subjective ratings can be biased. Indeed, rating errors frequently bias ratings and have plagued performance evaluations. Much of the performance appraisal (PA) research has focused on ways of eliminating, detecting, or controlling these rater errors. The results from these areas are mixed and insufficient in providing insights and understanding about how to deal with rater errors.;This research extends and tests a technique called differential person functioning (DPF; Johanson & Alsmadi, 2002) to the detection of rater bias (specifically leniency/severity) during a performance evaluation, as well as test a goal-based approach for performance evaluations. The DPF technique is used to identify the responses for a given individual that are different for different groups of items. The goal-based approach proposes that individuals' pursuit of different goals is what leads to different ratings. Two studies were conducted to examine these phenomena.;The first study was a pilot study to refine the materials and manipulations that were to be used in the main study. Specifically, two different evaluation formats were compared, sex differences were examined, and the manipulation was tested. In the second study (i.e., the main study) the sensitivity and consistency of the DPF technique was compared with two other traditional methods for detecting leniency/severity. Participants completed an actual performance evaluation for a faculty member under one of three different response instructions.;The results of the main study indicated that the DPF technique was not more sensitive than the other traditional methods. Indeed all methods examined were insensitive to the manipulation, thus all were ineffective at detecting rater bias. Although the DPF method was ineffective, results provided support for the goal-based approach. Those raters who were responding under different instructions (i.e., goals) provided significantly different ratings. These findings suggest that there was a reasonable opportunity for differential ratings to occur across groups, but yet none of the detection techniques were effective at detecting them. The discussions of these studies provide implications for the findings as well as implications for the DPF technique, the goal-based approach, and other personnel decisions. |
