Coefficient-based Regularized Algorithm For Estimating Gradient In High Dimensional Space

The 21 st century is a new era of big data.The Data contains more and more variables while it includes more and more redundant information.It's increasingly hard to learn from these data for statistical learning and machine learning.Therefore,variable selection is necessary before we establish learning models.The gradient of multivariate function is the vector whose components are the partial derivatives for each variable.Each component's norm corresponds to the extent of the function value's change when the variable in its place changes.Gradient estimation plays an important role in the problems of variable selection.So this article's target is learning gradient from the sample points.We propose a coefficient-based regularized algorithm for estimating gradient in high dimensional spaces.Compared with traditional gradient estimating methods,our algorithm does not need to partition the region of the variables so that it can be highly efficient in high dimensional space.We give the representer theorem for the algorithm which make it easier to get the solution.Moreover,by the singular value decomposition approach,we study how to reduce the matrix size in the representer theorem.An error analysis is given for the covergence of the gradient estimated by the reduced matrix size algorithm to the original algorithm.At the end of this paper,two examples are used to verify the effectiveness of our algorithm.The first experiment on man-made data set shows the validity of the algorithm.Then,in the second example,we analyze the numerical forecasting data of the city air quality and get the result which is accord with the actual situation.Both the examples show that our algorithm is effective and feasible.