A Unified Approach to Data Transformation and Outlier Detection using Penalized Assessmen

In many statistical applications normally distributed sample and sample without outliers are desired. However, in practice, it is often the case that the normality assumption is violated, such as when highly influential outliers exist in the data set, which will adversely impact the validity of the statistical analysis. In this dissertation, a Unified Approach is proposed to handle outlier detection, Box-Cox transformation using a penalized information criteria at the same time. This research started from investigating the performance of Box-Cox transformation in uncontaminated samples and suggested that the sample should be anchored to 1 before Box-Cox transformation is applied when the sample minimum is larger than 1. Simulation results showed that anchor-to-1 method is working well in enhancing the accuracy of Box-Cox transformation by decreasing the variance of lambda and eliminating extremely large or small values of lambda. The efficacy of Unified Approach is also verified in the clean samples including normal and lognormal, where the Unified Approach is able to tell that no Box-Cox is needed and no outliers are present. Later, simulations in the contaminated normal samples demonstrated that the Unified Approach can achieve the balance between a good model fitting (close to normal sample) and the complexity of data analysis through penalizing anchor-to-1, outlier exclusion, and Box-Cox transformation in the form of an adjusted information criteria. Through precise outlier detection and appropriate Box-Cox transformation, the efficacy of the Unified Approach is verified in the contaminated samples.