A systems approach to improving the quality of performance: An application to higher education

Graduate-student teaching apprentices (TAs) in the psychology department at a university in Michigan conducted seminars in an advanced undergraduate-level psychology course. All were members of the department's behavior analysis training system (BATS). BATS was an integrated series of courses and practicum experiences designed to achieve three main goals: recruiting, training, and maintaining the skills of competent behavior analysts. Twenty-five different tasks performed by the TAs were analyzed and measured prior to an intervention aimed at their improvement. Informal observation suggested that failure to complete these tasks reduced BATS' ability to meet its larger goals. Three well-known systems analysis models were applied, to differing degrees, in an effort to understand BATS as a system: Rummler and Brache's (1990) systems view to three levels of performance, Gilbert's (1978) behavior engineering model, and W. E. Deming's (1986) Total Quality Management (TQM) model. The application of these models to BATS was the first of its kind.; Two intervention packages were designed to enhance the reliability of TAs conducting the seminars and completing out-of-class assignments. These packages were also compared across semesters. In the first semester, the implementation of a combination of task checklists and face-to-face supervisor feedback was associated with statistically significant improvements in seminar and out-of-class performance of the TAs. The overall mean seminar completion percentage during the first semester increased from 70% during the baseline condition to 92% during the intervention condition. The overall mean out-of-class completion percentage increased from 63% during the baseline condition to 98% during the intervention condition.; These levels of improved performance were maintained during the second semester, with different tasks, when electronic-mail feedback was substituted for face-to-face feedback. The overall mean seminar completion percentage during the second semester was 96.9%. The overall mean out-of-class completion percentage was 98.5%.; In addition, two intervention packages were designed to increase the number of learn units (question-answer-feedback sequences) completed by TAs. In the first semester, face-to-face supervisor feedback had no statistically significant effect on the frequency of learn units the TAs completed in the seminars during the first semester. The overall mean number of learn units completed by TAs during the first semester was 50 per class during the baseline condition and 34 per class during the intervention condition.; However, during the second semester, videotape feedback added to the face-to-face supervisor feedback was associated with a statistically significant increase in the frequency of learn units. The overall mean number of learn units completed by TAs during the second semester was 21 per class during the baseline condition and 46 per class during the intervention condition.