Research On Loss Function In Machine Learning

With the development of technology, various data are much easier to obtain than they used to be. High dimensional and massive data are produced in various areas. How to deal with these data and extract useful information after screening the redundant information has become fundamental and the most important problem in statistical learning.Recently, the regularization methods which are efficient for high dimensional data analysis have been one of topics in machine learning. In this thesis, we study the theoretical properties of the regularization methods with different loss functions. The main work of this paper are the following:In chapter one, we review the background of the high dimensional analysis and regularization framework.In chapter two, we study the regularization framework with different loss functions, we give the bound of sampling error and generalization error for these loss functions.In chapter three, we focus on robust variable selection. We propose a vari-able selection method, LAD-Elastic Net, based on robust regression and Elastic Net which can deal with highly correlated data. We further prove the model selection consistency of LAD-Elastic Net.