Some Robust Estimation Methods And Application Examples Of Linear Regression Models

The linear regression models are widely used and the classic method of estimating regression coefficients is OLS. However, OLS is vulnerable to the outliers, i.e. it is not robust. Outliers almost always exist in real data. When outliers exist, using OLS is not good and prediction will be bad. Those limit the application of the linear models. Therefore, it is necessary to study ro-bust estimation methods of linear models. We mainly focus on reviews and comparisons of robust estimation methods of linear models.In the first chapter, we give the research background and framework. In the second chapter, we describe the basic idea, several concepts and de-velopment of robust statistics. In chapter3, we describe some robust es-timation methods of linear regression models:M-estimates, LMS-estimates, LTS-estimates, S-estimates, MM-estimates, FLS-estimates. We describe the definitions, algorithms, and some improved methods of these robust estima-tion methods. In chapter4, we give a brief introduction of several robust estimation methods for generalized linear models and highlight the Mallows pseudo-likelihood-estimates. In chapter5, we conduct numerical simulations. In Chapter6, we apply robust estimation to the credit scoring.In the section of numerical simulation, we add outliers through three ways:using different error distributions, adding outliers in Y direction di-rectly, adding outliers in Y and X directions. We use the following estimation methods:OLS, M, GM (generalized M), LMS, LQS (least quantile square), LTS, S and MM. And we get the following conclusions. When the errors follow a normal distribution, the robust estimation method is essentially as good as OLS. When the errors follow Laplace, Cauchy, contaminated normal distribu-tions, the robust estimation methods are significantly better than OLS. OLS is vulnerable to outliers. Robust estimation methods can be effective against outliers in Y direction. In the case of containing outliers in the X and Y di-rections, M method is not very good; GM is better than M; LMS, LQS, LTS and S are good. The efficiency and breakdown points of robust estimation methods are different. The contamination rate of the data and the degree of outliers are different. We can select the appropriate robust estimation method according to the actual situation.In the application example, we use MLE, Huber Quasi-Likelihood method and Mallows Quasi-Likelihood method. We compare estimation methods un-der different variables dimensions and different modeling samples. In the Mal-lows method, we use two x-weighted methods:hat method and MVE method. Quasi likelihood methods are significantly better than MLE. Mallows Quasi-likelihood method is better than Huber Quasi-Likelihood method.When out-liers exist, robust estimation methods are able to fit most of the data and thus make more accurate forecast.