Featere Augmentation Discriminant Analysis

Linear discriminant analysis is one of the most classical statistical learning methods.LDA is a charming method.On the one hand,compared with logistic regression,LDA provides robust estimates with higher efficiency of convergence,especially when the sample size is small.On the other hand,LDA is equal to Bayes optimal classifier under multivariate gaussian assumption,and central limit theory makes this truth extremely appealing.However,LDA fails for 3 scenarios:nonlinear decision boundary,multiple prototypes in one class and ultra-high dimension case.In this paper,we aim at solving the nonlinear decision boundary problem.We propose a procedure named Feature Augmentation Discriminant Analysis(FADA).Conditional Marginal densities of features proves to be the most powerful univariate classifier in Naive Bayes.Thus we construct new features with this kind of univariate transformation(log marginal density).Given a dataset splited into 2 parts,we use a part of data to estimate the conditional marginal densities of features for every class(?).Then,for the other part of data,we map the original features into higher dimensional feature space,by computing the log-marginal-density log (?).We finally solve a SDA classifier,which is a kind of robust discriminant analysis model in high dimensional case.Our method is a combination of FANS and SDA.While we get further,the binary classification strategy in FANS is generalized to multi-classification.Simulation and empirical study show that FADA not only provides an efficient strategy for nonlinear boundary problems,but also equips the model with a feasible visualization scheme.