| During the 1990s, most American states adopted content and student performance standards as a means to change pedagogy and improve student achievement. Some researchers suggest that the effectiveness of such policies depends in part on how the policies are interpreted and supported by local school leaders, including principals. Qualitative researchers have analyzed whether the desired reforms are being implemented and factors that support implementation, but the small sample sizes of their studies limit generalizability. Quantitative researchers have used large sample studies to investigate reform implementation, but their designs limit linking principals' beliefs and actions to teachers' practice. Large sample studies that correlate administrators' beliefs and actions with teachers' work are needed to help examine principals' roles in promoting teaching practices aligned with reform standards. “Small-q”qualitative research, that is coding responses to open-ended survey questions, potentially is well suited to this task because it can combine the richness of conventional qualitative methods with the generalizability of large-sample survey research, but reliable and valid methods for quantifying such open-ended survey items need to be developed.; As part of a larger study conducted by the Center for Educational Policy Analysis in New Jersey (CEPA-NJ), 83 elementary principals in New Jersey were surveyed about their work related to New Jersey's Core Curriculum Content Standards in mathematics and science and the Elementary School Performance Assessment (ESPA). Telephone interviews used open-ended and closed-ended questions. Drawing from research on the concepts of pressure, support, will, and capacity, three key variables were used to analyze principals' open-ended responses: district agenda, principal will, and principal agenda-setting. Using correlational analyses to examine the construct validity of these measures, their predicted relationships were assessed with variables derived from closed-ended responses from principals and teachers about teaching practices, standards-based reforms, assessment, teacher evaluation, parent/community involvement, and district factor groups. While a respectable degree of inter-rater reliability was achieved in coding the open-ended responses, correlational analyses provided only weak support for the construct validity of the variables derived from the open-ended questions. The findings chronicle the process and suggest issues and questions to be considered in combining qualitative and quantitative research methods. |
