The Analysis And Monitoring Of Data Models In Different E-commerce Rule Engines

Rule engine is a common technology in e-commerce decision framework.With the increase of data volume,rule engines and data models have become more complex and dynamic.To solve the challenge of monitoring run-time data models in rule engine upgrade phase,business analysts need a platform that can analyze and audit data models in big data environment.The thesis researches the current algorithm and architecture of e-commerce rule engines and addresses the current pain point for data model analysis.After researching and providing solution for current challenges,this thesis designs and implements a distributed analysis platform based on Spark.The implementation of the algorithm mainly consists of: raw data acquisition,data parsing,data analysis,aggregation and front-end reporting.Raw data acquisition adopts audit rule implementation to collect massive raw data.In parsing and analysis phase,the algorithm calls Scala and Spark RDD API to complete data analysis and aggregation before persisting the accumulated result into database.In front-end,the system leverages Spring framework for dynamic data pulling and demonstration.In project experiment,the platform leverages the feature of rule engines and data models to collect raw data in e-commerce big data environment.Spark in-memory computation and RDD feature enable highly efficient analysis of data models in different rule engines and improve auditing efficiency significantly.