| This study focused on the validity of the evaluation of teaching at a research university, particularly for representing the quality of an instructor's teaching. In the second part of the study issues of content validity, substantive validity, and consequential validity were studied, by examining: the meanings given to the construct, “good teaching;” the processes followed by undergraduate students when rating their instructors; and how race, gender and ethnicity affected the evaluation ratings of a sample of women and minority professors on campus. Data were collected over a period of two years using multiple methods, including participant observation, in-depth and focus group interviews, document analysis, and survey research. Data were gathered from students, professors, administrators, evaluators, professional developers, and attorneys.; The findings of the case study indicated that the evaluation system used by the department, although inclusive of multiple data sources, provided much less than a full understanding of the instructor's teaching. Almost no validity study of those inputs or review of all inputs as a whole had been conducted. Other findings indicated that student ratings were defined and constrained by the items and procedures the Department used. Peer evaluation by members of the Executive Committee, when shared with the instructor, was constrained by colleagueship, but also shaped by a need to maintain a collective image of good teaching across the Department, partly to support its research mission.; Findings of the second part of the research indicated that students differed in defining good teaching and followed different processes in assigning ratings to their instructors. Other findings indicated that student motivation towards the course content, the ability of the instructor to speak English well, and fairness of grading were among the most influential influences on the student ratings. In addition, class size, course level and the “required” or “elective” nature of the course also influenced student perceptions of the instructor's teaching. The study also found some evidence of negative influences of race, gender and ethnicity of the instructor on the ratings that students gave their instructors. |
