The Design And Implementation Of The Data Analysis Engine In The Distributed Tracking System

As business continues to expand,traditional stand-alone architectures have been unable to meet business needs,and applications are often implemented with complex,large-scale distributed architectures.However,when there is a failure,the operator find it hard to locate the service node in problem.The distributed tracking system can track the user requests and restore the service call topology under the complex distributed environment,and provide a practical solution for the operator to locate the fault service node quickly.But the data collected by the distributed tracking system has the characteristics of relevance,disorder,real-time,burstiness and concurrency,These characteristics bring great challenges to the data analysis engine.When the data is high concurrency,it can easily cause blocking and even loss if the data processing is too slow.And adopting the method of data full storage not only brings great pressure to data storage,but also results in inefficient data association query.Therefore,this dissertation presents a design and implementation scheme of data analysis engine in distributed tracking system.In order to solve the problem of data blocking or loss due to mismatch of data reception and processing speed,the data analysis engine designed a data receiving and processing separation analysis model.The analysis model not only guarantees the data receiving ability,but also guarantees the load balance of the processing nodes.Then,the data processing node is introduced into the Redis cache to analyze the application invocation relationship in real time,and directly save the call relationship to the database to solve the problem of the data association query slowly.At the same time,data pre-aggregation method reduces the pressure of data storage.The Redis cache is optimized for high performance.Finally,the experimental results show that the data analysis engine has good data processing ability.