Key Technology Research Of The Meteorological Monitoring Data Packets Parallel Processing

In recent years, meteorological satellite monitoring data packets(Meteorological message for short)are showing explosive growth, and the transmission of meteorological messages has a feature of burstiness,the requirement of real-time data processing is becoming more and more higher. The use of high-performance clustering meteorological to monitor data can achieve good results, but there are verying degrees of problems of nodes load imbalance exist in current scheduling algorithms. To solve these problems, this paper deeply studies the classical weighted least connection scheduling algorithm and put forward a dynamic priority scheduling algorithm which is based on the performance and load of the hardware nodes, then take the scheduling algorithm as the core and design distributed parallel processing model for meteorological message. Experiments show that the parallel processing model has a better load balancing effect.The main work and contributions are as follows:(1) Study the current scheduling algorithm, for the problem of load imbalance exists in these scheduling algorithm, we improve appropriately the weighted least connections algorithm, and propose a new load balancing algorithm. When the amount of tasks reaches a certain number,the improved load balancing algorithm has less average task response waiting time and the more the number of tasks is, the more obvious the effect is.(2) Based on the improved task scheduling algorithm, the distributed parallel processing model of meteorological message is designed according to the characteristics of the meteorological message with the same packet format and the processing time. A large number of tests show that the proposed modelsignificantly improves the throughput of processing data, makes full use of the resources of each node, and reduces the average waiting time of the task.(3) The model has been successfully applied to the new generation of civil aviation meteorological information processing and transmission system, and has passed the acceptance of the expert group.