Research On Sequential Change Detection And Isolation With Applications In Parallel Data Streams

Change detection problem is aimed at the detection of the change of the state of the random process.The change point here corresponds to the moment when the probabil-ity distribution of the data stream changes.The decision maker needs to judge whether the probability distribution of the data stream changes in real time according to the data observed sequentially.When there are multiple possible post-change distributions,we need to further diagnose the correct post-change distribution.With the rapid develop-ment of modern sensor technology and computer technology,people can simultaneously obtain many real-time data streams of sensors in different fields or regions.Therefore,it is no longer a single-stream change detection problem,but will be extended to the multiple change detection problem.This paper mainly formulates and investigates the problem of sequentially detecting changes in parallel data streams.Each data stream may have its own change point at which the underlying probability distribution of its data changes,and the decision maker needs to declare,sequentially,which data streams have passed their change points.For parallel sequential change detection and isolation problems,the traditional per-formance metrics for the single-stream change detection and isolation problem are not applicable to the multi-data stream scenario.Therefore,we creatively apply the error metrics in multiple hypothesis testing,false discovery rate(FDR),which is the expect-ed ratio of the number of falsely declared data streams to the total number of declared data streams,to our research problems.Based on the relevant results of single-stream change detection,we present the performance analysis of the proposed decision proce-dures in parallel data streams,and prove its false detection performance controllability and scalability for large-scale data streams.In addition,we prove that a new Bayesian decision procedure is asymptotically optimal for single-stream change detection and isolation problems.The main contributions of our work are as follows:1)We study the change detection within parallel data streams from non-Bayesian and Bayesian situations,respectively.In the non-Bayesian case,we formulate the change detection for multiple data streams and develop an FDR-controlled decision pro-cedure to detect all finite changes.For the developed FDR-oriented decision procedure,we establish a theoretical guarantee for the FDR.Furthermore,we obtain the asymptot-ic behavior of the average detection delay(ADD)as the number of data streams grows large,which does not grow with the number of data streams;in contrast,the ADD of familywise error rate(FWER)-oriented decision procedures grows logarithmically with the number of data streams.We test the developed decision procedures for simulated data sets to corroborate the analytical results.In the Bayesian case,we formulate the problem of change detection for parallel data streams taking into account truncation of data.The FDR metric is extended to this problem,and a sampling restricted Bayesian decision procedure is proposed.The analytical results show that the ADD of the FDR-controlled decision procedure is much better than that of FWER-controlled procedures.Finally,we conduct a case study in which we apply the proposed decision procedures to the multichannel dynamic spectrum access problem for cognitive radios,and further illustrate the utility of the decision procedures.2)To cope with the scenario where the post-change statistics are under multiple possible hypotheses,we extend the problem formulation of change detection to change detection and isolation,for parallel data streams,and develop the corresponding deci-sion procedures and performance guarantees.With the help of the theoretical results of single-stream change detection and isolation,we prove the controllability of the FDR-oriented decision procedure and obtain the asymptotic behavior of the ADD.By means of simulated data sets,we explored the influence of different parameters of the formu-lation on the performance of decision procedures.Finally,we conduct a case study in which we apply the proposed decision procedures to online monitoring of abrupt price changes in the stock market,and confirm the practical feasibility of the proposed deci-sion procedures.3)In parallel sequential change detection and isolation problems,since the deci-sion algorithms adopted do not make use of the prior knowledge of change points,we believe that this prior should be utilized to find a decision algorithm with better perfor-mance for the Bayesian change detection and isolation problems.Thus,we formulate the problem of Bayesian sequential change detection and isolation with the general pri-or of the change point,and propose new criteria of optimality.Then we develop the corresponding decision procedure,and analyze its statistical properties,for which we establish upper bounds on the probabilities of false alarm and false isolation,and an asymptotic upper bound on the ADD.We derive an asymptotic lower bound on the AD-D of all decision procedures that satisfy the proposed error constraints.Thus we prove that our proposed decision procedure is asymptotically optimal under certain regularity conditions.