Research On Restricted Biased Estimation And Preliminary Test Estimation Of Parameters In Linear Models

Linear models play a central part in modern statistical methods and they have become one of the most widely used classes of models in modern statistics. In this dissertation, we mainly focused on the restricted biased estimation and preliminary test estimation of parameter in linear models with equality restrictions and stochastic linear restrictions.For linear model with exact linear equality restrictions, by respectively combining the ridge estimator and Liu estimator with the restricted least squares estimator (RLSE), a new class of restricted ridge estimator and restricted Liu estimator which always satisfy the given linear restrictions are proposed. The new restricted ridge estimator is proved to be superior to the traditional ridge estimator and the RLSE, while the new restricted Liu estimator outperforms Liu estimator and the RLSE in the sense of mean squared error matrix (MSEM). In addition, this paper generalize the r-k estimator and r-d estimator proposed recently in literatures to the cases with restrictions, namely we have studied the restricted r-k estimator and restricted r-d estimator in this paper, and we similarly prove that the restricted r-k estimator and the restricted r-d estimator outperforms the r-k estimator and r-d estimator in the MSEM sense, respectively. For the Liu-Type estimator considered by many researchers in literatures recently, we have made a further discussion about the choice of the tuning parameter and some other fitting characteristics. In particular, we derived two methods to determine the optimum tuning parameter, which are to maximize the coefficient of multiple determination or to minimize the generalized cross validation (GCV) of the prediction quality. It's proved that for the Liu-Type estimator, the ridge parameter could serve for regularization of an ill-conditioned design matrix, while the tuning parameter could be used for tuning the fit quality effectively, and as the ridge parameter increases, the Liu-Type estimator produces more robust regression models than the ridge estimator. Numerical examples are given to illustrate the theoretical results.For linear model with stochastic linear restrictions, in this thesis we generalize the traditional ordinary mixed estimator (OME) to singular linear model and proposed singular mixed estimator (SME). Performances of the SME in the MSEM sense and its two-stage estimator are also discussed. Applying the SME in the parameter estimation of Panel data model with stochastic linear restrictions, we derived four feasible estimators for Panel data model. The relationship among them is discussed and it's shown that they are superior to the corresponding unrestricted estimators. Furthermore, we have studied the prediction problem in singular linear model based on the SME. The optimal heterogeneous predictor, optimal unbiased homogeneous predictor and the optimal predictor for singular linear model with stochastic linear restrictions are derived, and we find that all the three predictors satisfy a general formula for prediction. By combining the idea of mixed estimation and Liu estimator, we proposed a new stochastic restricted Liu estimator and prove that it outperforms the traditional Liu estimator, OME and some other stochastic restricted estimators under certain conditions. Simulation study and numerical example have supported the theoretical results derived. For two common types of misspecification of regression models, we have further studied the behaviors of the stochastic restricted Liu estimator in such two cases, and performances of the corresponding predictors are also examined.For the preliminary test estimator when exact linear restrictions are used, we have firstly discussed some common test methods in literatures, including the familiar F test and some large sample tests widely used in Econometric models such as the Wald test, Likelihood Ratio (LR) test and Lagrangian Multiplier (LM) test. By combing the idea of preliminary test and Liu estimator, we have proposed three preliminary test Liu estimators (PTLE) based on the Wald, LR and LM tests. Through the bias analysis, we find that the Wald test based PTLE has the smallest quadratic bias,followed by the PTLE based on the LR and LM tests. On the other hand, we find that when near the null hypothesis, the LM test based PTLE has the smallest risk, followed by the PTLE based on the LR and Wald tests, while when the parameter departs from the restrictions, the situation is reversed. In the meantime, when the Liu parameter is small, then the PTLE based on W test has the smallest MSE followed by the LR and LM tests, while when the Liu parameter is large and near one, the situation is just also reversed. Furthermore, the relative efficiency and the choice of optimal significance levels are also discussed. Considering the fat-tail phenomenon that may exist in practical data, we have also studied the PTLE for the multi-t distribution model based on the three large sample tests above. Performances of the estimators according to the quadratic bias and mean square error are similarly compared in detail.Finally, we have also considered the preliminary test estimator when stochastic restrictions are used in regression. By imbedding the stochastic linear restriction model in an exact linear restriction framework and make use of the results concerning the quality restricted estimator, we propose the stochastic hypothesis preliminary test ridge estimator (SPTRE) based on F test. Performances of the SPTRE, stochastic hypothesis preliminary test estimator and the ridge estimator in the sense of MSE are systematically analyzed. Relative efficiency of the SPRE and the corresponding optimal choice of the significance level are also discussed in this paper.