Dimension reduction in regression analysis

Dimension reduction in regression analysis reduces the dimension of the predictor vector p-dimension x without specifying a parametric model and without loss of information on the regression of y on x. The problem is how to identify the low dimensional projection of x. The projection can be represented by the k eigenvectors corresponding to the nonzero eigenvalues of a k rank p by p matrix with ( k < p).;In this dissertation, we introduce the concepts of a target matrix , which is a population p by p matrix to be estimated, and estimation methods which estimate the target matrix from data. We present a new perspective on three existing methods, SIR (Li, 1991), SAVE (Cook and Weisberg, 1991) and pHd (Li, 1992). Their target matrices and estimation methods are identified and distinguished. A system is built to identify and construct more target matrices and therefore more potential methods. SIR, SAVE, and pHd are unified as special cases of a broad new class of methods. In particular, we propose methods based on linear combinations of known target matrices.;Because there are now so many methods of dimension reduction, we introduce methodology to select among different target matrices and estimation methods. A k-dimensional estimate is the first k eigenvectors of an estimated target matrix. The variability of the estimate is defined and assessed using a resampling plan.;In general, no target matrix is guaranteed to identify the entire k-dimension projection of x. Assume the response y is categorical and the predictors x given y are normally distributed, the SAVE target matrix is shown to be able to recover the entire k-dimension projection. We introduce Bayesian modeling techniques as a new estimation method to estimate a target matrix, and study the behavior of various posterior estimates of the SAVE matrix with different priors. Examples with small sample size illustrate that Bayesian techniques can be useful.