Renewable Quantile Regression Estimation Methods For Large Scale Streaming Datasets

Streaming data analysis has drawn much attention in the past decades,where large amounts of data arrive in streams and fast analysis without access to the historical data is necessary.Quantile regression has been widely used in many fields,with robustness and comprehensiveness.In the streaming data environment,it is difficult to implement quantile regression by using the conventional methods,because they all require to process the entire data set together.To fix this issue,this paper proposes a novel online renewable quantile regression strategy.we only need to retain the local quantile regression estimator and a weight matrix for each data stream,and the historical data can be completely discarded.Then the new estimator can be efficiently renewable as the minimizer of a weighted least squares type loss function,which is constructed by the local quantile regression estimators and weight matrices.Therefore,computation efficiency is high,and there is no need to store original historical data.The second section of the updated quantile regression estimation method has strict requirements on the local data volume.When the local data volume is small or the number of data streams is large,its statistical properties cannot be guaranteed.Moreover,the non-smooth quantile regression loss poses new challenges in both computation and theoretical development.In order to solve these two problems,we first introduce a convex smooth quantile regression loss in the third section,,which is infinitely differentiable and converges to the quantile regression loss uniformly.Then an online renewable framework is proposed,in which the quantile regression estimator is renewed with current data and descriptive statistics of historical data.The renewable smooth quantile regression solves the defects of the second section of the method,and the theoretical results also ensure this..