LARS Diagnostics Regression Tree

Two popular methods for classification and regression are linear regression and tree induction, which have some what complementary advantages and disadvantages. Linear regression is a powerful technique for fitting a simple model to the data, and the process of model fitting is quite stable, resulting in low variance but potentially high bias. But linear regression models are difficult to interpret if collinearity, nonlinearity, or interactions are present. On the other hand, Tree methods exhibits low bias but often high variance. It searches a less restricted space of models and captures nonlinear patterns in the date, but it's less stable and prone to overfitting. So this article proposes to fit a piecewise linear regression model by recursively portioning the data and fitting a different linear regression in each partition, named LARS Diagnostics Regression Tree (LDRT).Firstly LDRT propose binary tree-based procedures to check the adequacy of linear function between predictors and the target variable and group the predictors into four types: n-variable, f-variable, s-variable and c-variable. Then LDRT employs the growing tree method of the GUIDE least regression tree algorithm avoids the variable selection bias employing a two-step approach to split selection. LDRT obtains estimates and prediction subject to constraints on the coefficients representing the effects of splits in the tree. In order to get the fast training speed, LDRT employs LARS method to shrink the nodes of the initial tree and select the best-sized subtree. The procedure leads to both shrinking of the node estimates and pruning of branches.Finally, we explore and illustrate the performance of LDRT via Monte Carlo Studies. Compared to CART and GUIDE, LDRT algorithm is very fast training speed, easy to interpret, stable and accurate prediction.