Exploring the use of statistical process control methods to assess course changes

This dissertation pertains to the field of Engineering Education. The Department of Mechanical Engineering at the University of Nevada, Reno (UNR) is hosting this dissertation under a special agreement.;This study was motivated by the desire to find an improved, quantitative measure of student quality that is both convenient to use and easy to evaluate. While traditional statistical analysis tools such as ANOVA (analysis of variance) are useful, they are somewhat time consuming and are subject to error because they are based on grades, which are influenced by numerous variables, independent of student ability and effort (e.g. inflation and curving). Additionally, grades are currently the only measure of quality in most engineering courses even though most faculty agree that grades do not accurately reflect student quality.;Based on a literature search, in this study, quality was defined as content knowledge, cognitive level, self efficacy, and critical thinking. Nineteen treatments were applied to a pair of freshmen classes in an effort in increase the qualities. The qualities were measured via quiz grades, essays, surveys, and online critical thinking tests. Results from the quality tests were adjusted and filtered prior to analysis. All test results were subjected to Chauvenet's criterion in order to detect and remove outlying data.;In addition to removing outliers from data sets, it was felt that individual course grades needed adjustment to accommodate for the large portion of the grade that was defined by group work. A new method was developed to adjust grades within each group based on the residual of the individual grades within the group and the portion of the course grade defined by group work. It was found that the grade adjustment method agreed 78% of the time with the manual ii grade changes instructors made in 2009, and also increased the correlation between group grades and individual grades.;Using these adjusted grades, Statistical Process Control (SPC) methods were employed to evaluate the impact of the treatments applied to improve the courses. It was determined that using SPC is advantageous because it does not require additional resources and is not affected if a course is curved by adding the same amount of points to each student's grade. It was also determined that SPC results, unlike average grade, correlated well with anecdotal evidence from the instructors concerning how well the students performed in any given year. In addition to application of SPC to evaluate curriculum change, statistical analysis was used to show that course grades correlate with quiz grades, but do not correlate with critical thinking, self efficacy, or cognitive level which implies that treatments need to be implemented to increase these qualities.