Research On Abnormality Monitoring Based On Event Stream Computing

The event stream computing technology is a new type of big data processing technology,which combines event processing technology and stream computing technology.It can not only provide large-scale real-time online data processing,but also can be applied to scenarios with certain logic judgment capabilities,such as scene recognition and intelligent processing.The event stream computing system is a multicomponent collaborative large-scale distributed real-time system,which is characterized by complex components,abnormally high,and small sample.Therefore,research on anomaly monitoring methods for event stream computing system plays a crucial role in ensuring reliability and security in the field of big data processing.Based on a comprehensive analysis of the working state of the event stream computing system,this thesis proposes an anomaly monitoring scheme according to the characteristics of the event stream computing system.The main research contents of this thesis are as follows:1.Aiming at how to describe the behavior characteristics of event stream computing system and how to adaptively monitor abnormal conditions online,this thesis proposes an abnormality monitoring method based on the fast search clustering center algorithm for event stream computing system.First,by analyzing the event stream computing system behavior characteristics,combined with the event state space and the physical state space of the event stream computing system,a method can be constructed to describe the system behavior characteristics,and the system behavior state is characterized.In addition,analyzing the anomaly monitoring of the event stream computing system has the characteristics of online,adaptive and multi-point monitoring,and considering the limitation that the fast search clustering center algorithm can not automatically determine the number of clusters,a combination of the fast search clustering center algorithm and the outward statistical testing algorithm for abnormal monitoring of event stream computing system is proposed to realize automatic identification of the system's beheavier status.Finally,this thesis introduces an adaptive anomaly monitoring framework to automatically adjust the network structure and size and then achieve online adaptive anomaly monitoring.In order to verify the proposed method,this thesis uses the real data set to carry out simulation experiments.The experimental results show that the method can describe the behavior of the system well and effectively realize the abnormal monitoring of the event stream computing system.2.For the limitation of the fast search clustering center algorithm can not make full use of the category information in the data,and the advantages of support vector machine classification for small sample data,an abnormality monitoring method of the event stream computing system based on support vector machine is proposed.First,due to the high dimensional and nonlinear characteristics of the event stream computing system feature space,an improved local linear embedding(SKLLE)algorithm is proposed to reduce the dimension and realize the linear output of high dimensional space in low dimensional space.Second,the support vector machine is used to train the abnormal behavior of the system,and the multi-anomaly classifier is obtained to realize the identification of the abnormal information of the event stream computing system.Finally,the feedback mechanism is used to continuously iterate and update the multi-anomaly classifier to realize the adaptive anomaly monitoring of the system.Experiments show that the proposed method has strong robustness and self-adaptive ability for event stream computing system anomaly monitoring.