Acting rationally with incomplete utility information

Rational decision making requires full knowledge of the utility function of the person affected by the decisions. However, the task of acquiring such knowledge is often infeasible due to the size of the outcome space and the complexity of the utility elicitation process. We argue that a person's utility value for a given outcome can be treated as we treat other domain attributes: as a random variable with a density function over its possible values.; We show that we can apply statistical density estimation techniques to learn a probabilistic model of utilities from a database of partially elicited utility functions. The Bayesian learning framework we define for this problem also allows us to discover the number of statistically coherent subpopulations and select a structured utility model for each subpopulation. The factorization of the utilities in the learned model and the generalization obtained from density estimation allow us to provide a compact and robust representation of the utility functions in the population. Such a model can serve as a prior distribution when we encounter a new user whose utility function we need to discover. Any information we obtain in the course of utility elicitation process or by observing the user's behavior can be used to adjust the model to the user by Bayesian conditioning.; We present two applications of such customizable utility models. The first, designed for a decision support system, directs the utility elicitation process by choosing questions most relevant for the current user and the current decision problem. The second focuses on non-cooperative situations where a second agent is optimizing its own decisions relative to its uncertainty about the utility of the first.