Research On Distributed Real-Time Analysis Technology Of Full-View Synchronized Measurement System

The Full-view Synchronized Measurement System(SYMS)enables all-round real-time testing of new power systems with a high proportion of power electronics.However,due to the large number of data types,the proliferation of devices and the complexity of the data analysis process,it is difficult for a centralised SYMS master to guarantee the real-time and reliable processing of synchronous phase data.The thesis proposes a distributed architecture for the SYMS master station,investigates the distributed real-time analysis method based on the stream processing framework of Apache Storm adapted to multilanguage and multi-time window algorithms,and designs and implements a twotier online load balancing scheme based on traffic sensing and consistency hashing techniques,finally building a distributed real-time application platform for the SYMS master station.In order to address the problems of single point of failure,low concurrent processing capability and low scalability of the traditional wide area synchronous measurement system master architecture,the data application requirements,storage requirements and hardware resource requirements of the current master application platform are analysed in detail,based on which a distributed system architecture for the SYMS master application platform is proposed that includes real-time application services,data caching services,offline application services and web services The data services are deployed in different servers,and communication is coordinated through messaging,reducing the coupling between the data services,and the single data service can be clustered and extended,which not only enhances the fault tolerance of the application platform,but also ensures its concurrency and scalability.To address the problems of high latency and multiple types of algorithms for online analysis of master data,a Kafka-Spout-based asynchronous communication method and a distributed computing method based on the stream processing framework of Storm adapted to multi-language and multi-time window algorithms were designed and developed.Based on this,a large-scale Storm computing cluster was built in conjunction with the SYMS master private cloud to build a distributed real-time application platform for the SYMS master.This enables multi-level analysis of large-scale real-time phase data with low latency at the second level,effectively improving the SYMS master’s monitoring of the operational status of new power systems.To address the problems of blocked computing resources and extended computing time caused by the concurrent access of massive data from SMD devices to the main station application platform,the principles and application scenarios of load balancing technology are first analysed,followed by a decomposition of the current operational problems of the SYMS main station application platform to find suitable load balancing solutions respectively.Combining the characteristics of SYMS master,a scheduling algorithm based on the optimal parallelism greedy policy is studied,and a two-tier online load balancing scheme based on traffic-awareness and consistency hashing is proposed,which not only achieves timely load balancing for heavily loaded nodes,but also enables the Storm cluster to perform elastic computing.This effectively reduces the overall load pressure on the cluster and also further shortens the failure detection time of the SYMS master.Finally,a load test was conducted on the complete load balancing solution to verify the effectiveness of the solution.