Statistical Inference And Rapid Calculation Of Mean Change Point In Massive Data Set

The problem of change-point detection has always been one hot issue in the field of statistics.It has been widely used in quality control,earthquake disaster prediction and other fields.Existing change-point analysis methods rarely take into account the computational complexity,memory requirements and privacy issues under large data size.In this paper,we propose two kinds of fast estimation methods: centralized estimation method and distributed estimation method.In the case where the amount of data is huge but can be processed collectively,the least squares estimation of the mean change point has a high computational complexity,so it is necessary to reduce the computational complexity.In this paper,a three-stage fast scanning algorithm is proposed for the estimation of mean change point,and it is proved that this method has the same convergence speed and limiting distribution as the least squares estimation of mean change point,and reduce complexity.We have conducted sufficient data experiments in terms of computation time and estimated efficiency,and the results show that the estimated efficiency of the new and old methods is similar,but the computation time of our method is obviously shortened.When data cannot be processed centrally,based on a subsequence data stored in one single machine,we get a change-point pre-estimator which is used to construct an interval covering the true change point with large probability,and then search the change-point more precisely on this interval among all machines.The final estimator by the above algorithm is proved to have consistency and limiting distribution with the same performance under the data-centralized case.The effectiveness of our algorithm is verified by sufficient numerical experiments which show that the asymptotic properties of our method are very close to that of traditional one,but with much less computation time.