Active modeling in cost-sensitive environments

Machine learning of predictive models has emerged in recent years as an increasingly effective mean for capitalizing on an unprecedented availability of information, allowing for the learning of models that can be used to respond to emerging trends in a timely and efficient manner. Of particular interest in this thesis are models for class probability estimation, given their key role in the evaluation of alternatives in cost-sensitive environments.; In many cost-sensitive environments, the acquiring of labeled examples for learning incurs significant costs. This thesis studies methods that reduce the costs of acquiring labeled examples for learning Class Probability Estimation (CPE) models to support objectives such as CPE accuracy and accurate decision-making. The primary contribution of this work is a novel active learning framework. Within this framework, new generic algorithms have been developed that identify particularly informative examples for learning accurate CPE models, and models of CPEs that promote decision-making accuracy as well as classification accuracy.; The BOOTSTRAP-LV method is developed here to reduce the costs of learning accurate CPE models. A comprehensive evaluation of the method against alternative approaches demonstrates that it provides significant economies over a wide range of domains and for different model learning schemes. The analyses that follow promote further understanding of the instrumental role served by two fundamental principles underlying the proposed framework, which govern the choice of informative training instances. The empirical analysis also investigates the effect of the algorithms' parameters on its performance.; Making decisions in cost-sensitive environments often means resorting to decision-theoretic approaches for evaluating alternatives, requiring the estimation of expected benefits of various courses of action. The GOAL algorithm that is developed here economizes on learning costs of models that promote accurate decision-making. The empirical analysis that follows reveals interesting tradeoffs between costly learning of accurate CPEs and decision accuracy.; An instantiation of the proposed framework for learning accurate classification models was also developed. The Weighted Uncertainty Algorithm reduces the cost of learning classification models. An empirical evaluation demonstrates that it improves the performance exhibited by an existing generic active learning algorithm.; The comprehensive analyses and valuations presented in this thesis provide ample evidence for the efficacy of the proposed methods and important insights regarding the properties of existing methods and of those developed here, advancing understanding of the application of active learning methods for different tasks.; With the ubiquity of automated modeling in many economic settings, cost-sensitive learning of predictive models is becoming an increasingly critical task. In addition to its contribution to the machine learning literature, this work contributes to modeling practices in such cost-sensitive environments, and the algorithms developed here allow modeling to be employed more intensively, by minimizing the associated costs, and by improving the economic desirability of these practices.