| A selection procedure consisting of both multiple-choice (MC) paper-and pencil and hands on performance assessment (PA) elements was developed for a large consumer products manufacturing company in the southeastern United States for the purpose of ensuring that workers possessed the necessary knowledge, skills, and abilities required for work at a new facility. Two 125-item alternate tests using an MC format and 7 PA exercises were initially developed for the job referred to in the present study as L2/L3 Production Technician. The purpose of this study was to examine the validity evidence for the two alternate multiple-choice (MC) job knowledge tests and seven performance assessment (PA) exercises that were developed for employment selection purposes. The study sample included 432 Form A and 324 Form B examinees who took both the MC test and the PA exercises. Factor analysis results revealed that the same construct, labeled as applied mechanical knowledge, was measured by both the MC tests and the PA exercises. Item and test analysis results supported the use of Form A and Form B as alternate test forms. The decision consistency between the MC tests and the PA exercises did not appear to be sufficient to recommend that either form of the MC test alone could be used to select qualified L2/L3 Production Technicians. The correlations between MC score and PA total score were .627 for Form A and .612 for Form B. As part of a content analysis, subject-matter experts rated a large number of MC items as either having "no relationship" or "small relationship" to the PA exercises. However, subject-matter experts did rate the PA exercises as having a great importance to the job of L2/L3 Production Technician. |
