A Security Practices Evaluation Framewor

Motivation. Security breaches, and the software vulnerabilities that enable them, have increasingly become headline news. Avoiding or resolving vulnerabilities during software development has become a consideration for many organizations that build and ship software.;Research Problem. Many software development security practices have been recommended. However, empirical evidence on the application and results of the practices is limited. The goal of this research is to support researcher and practitioner adoption of security practices by developing a model for how security practice adherence affects security outcomes and a framework for the collection of software development context factors, practice adherence, and security outcomes.;Approach. To support the collection of empirical evidence for the effects of security practice use in software development, we propose a model, the Security Outcomes Theoretical Model (SOTM), for how security practice adherence affects security outcomes, supported by a measurement framework, the Security Practices Evaluation Framework (SP-EF). SOTM is a set of constructs and relationships that embody a theory of how security practices affect security outcomes. SP-EF is a measurement framework that includes repeatable instructions for identifying the use of these practices, recording the context of the team using the practices, and measuring the security outcomes for the software.;Results. We conducted a literature review to identify software development security context factors, security practices, and outcome measures. We identified a set of security practices, practice adherence metrics, outcome measures, and context factors, and assembled them as a measurement framework, SP-EF. We conducted a case study in which we collected all SP- EF data from an industrial software development project. We found agreement between the researcher and team views of security practice use on the project, and evaluated the effectiveness of automated means of assessing practice adherence. We identified use of all of the practices specified in SP-EF by one or more survey participants. We conducted a survey of open source development projects to assess the use of the SP-EF security practices and the degree to which our adherence measures correlate with security practice use. We found empirical support for the use of the SP-EF security practices. We found that Training has a positive, statistically significant correlation with Usage, suggesting that investment in training supports practice usage. Finally, we assessed whether the theorized relationships in SOTM hold in observational data by combining SP-EF measurements available in published datasets. Our data suggest that assessing the state of software security requires accounting for both software development context factors and the software usage context factors.;• The Security Outcomes Theoretical Model, a proposed model of the constructs affecting software development security outcomes;;• SP-EF, a proposed set of measures for assessing security in the software development process including instructions for data collection, and.;• Empirical evaluation of the proposed model and metrics using two open source datasets.