The Research On Predictive Complex Event Processing Technology For Streaming Big Data

In the era of big data,a large amount of data is generated from the Internet,sensor networks,social networks,etc.The less time it takes to make decisions in the face of numerous pieces of information,the more valuable information that can be obtained from the entire process,therefore,streaming big data processing technology has attracted more attention.In practical applications,such as intelligent transportation,health care,and financial risks,users want to dynamically process the complex events identified from the primitive event stream,count the most relevant and meaningful parts of the event,and effectively i nfer the predecessor events or successor events that may occur when a given event occurs.However,the current predictive analysis methods for complex event processing are not mature enough,there are still many challenges in the field of streaming big data,and further research is needed.In order to solve the current challenges and problems,this paper uses Bayesian network model to study complex event processing technology,the Bayesian network model has a solid mathematical foundation and it is a model that can effectively represent uncertain knowledge and reasoning.In view of the concept drift of data distribution,this paper proposes different model construction methods for the gradual change of data and sudden change of data,which can better meet the real-time changes of data distribution in the real environment and achieve the best predictive analysis results.The main work of this paper is as follows:First,aiming at the problem that fixed models do not perform well in the case of data gradation in streaming big data environments,a predictive complex event processing method based on evolving Bayesian networks is proposed.The Bayesian network model is designed based on event type and time and the inference method is Gaussian mixture model and EM algorithm,the evolution process of the model is based on BDe metric and MMHC algorithm.When learning the Bayesian network structure from the event stream,this method supports the incremental computing scoring metric when new data arrives or the edge of the network structure changes,and evolves the Bayesian network structure based on the h ill-climbing algorithm.The system can continuously monitor the Bayesian network model and modify it in time if it does not match the new incoming data to support the dynamic update of the network model.The experimental results show that this method has higher accuracy and is suitable for the prediction of complex events.Second,aiming at the problem that the progressively evolved model may not be able to learn a suitable model in a short time in the case of sudden change of data,a predictive complex event processing method based on a structure varying dynamic Bayesian network is proposed.This paper uses complex event processing and event context as the basis of the method,it divides historical data by offline context clustering to obtain different clusters,and updates clusters when clustering event streams online.The data in each cluster is evaluated by a score-search method to learn the corresponding Bayesian network structure,and Gaussian mixture model is used for approximate inference.When predicting online data,the appropriate Bayesian network model or combination of models for the current context is selected for prediction,and the model is updated in real-time during the prediction process.The experimental results show that this method has better feasibility than the current popular methods.