| Railway power supply system is an important equipment for railway transportation,which is responsible for the power supply of electric locomotives,EMUs(Electric Multiple Units),stations along the way,communications,illumination,signal lamps and interlocking devices and other equipments,safe and reliable power supply directly influences the safety of railway transportation.Thus to ensure safe power supply,the railway sector installs railway power supply monitoring system.Since the railway traction power supply system is a typical dynamic electric network,its electric locomotive load is impact load with high power,and it also holds characteristics of large fluctuation range and rapid change and so on.In addition,the high speed and high running density of EMU accelerate the changes of running parameters of the railway power grid,such as voltage and current,and the controlled terminals will collect a large amount of on-site real-time data and upload them to the dispatching center.Besides,the dispatching monitoring system needs to carry out access processing on those information continuously,and long term operation would generate large amount of information.At present,the data storage platforms of the monitoring system are mostly based on relational database,whose storage capacity is generally restricted at TB level,when faced with rapidly increasing,dynamic and numerous monitoring data,it has poor loading and query performance.If carry out access processing on those information,the response speed of the system is slow,or even there would be shutdown,information would thus be delayed,this would not only affecting the real-time performance of the system,when serious,this would even cause delay or omission of key information and result in delay or disappearance of alarm,all those would directly threaten the safety of power supply and could not satisfy demands of storage and processing of increasing number of information.Therefore,how to quickly store and process massive monitoring data is a key issue urgently needs to be addressed.Aimed at the storage and query difficulty of massive monitoring information in railway power supply monitoring system,this paper integrates Hadoop,Hive and Impala big data cloud computing components,establishes railway power supply dispatching and monitoring cloud computing cluster,with Beijing EMU section 10 kV power remote monitoring system as the input data,research on the distributed Map compression storage,HQL Query Optimization and Impala fast query of dispatching monitoring data have been carried out,to realize highly-efficient compression storage and rapid query of mass monitoring data.The research results demonstrate that:Firstly,after applying the distributed Map compression on the monitoring data the compression and import speed is improved significantly,and data volume is reduced greatly,at the same time,there is nearly no influence on the query.What's more,the Map_Gzip,Map_Deflate,Map_Snappy and Map_LZO formats accelerate the query speed on the contrary,which provides a new possible solution to solve the problem of massive monitoring data access.Secondly,HQL join query time is significantly reduced after adoption of distributed Map compression at the query shuffle phase,among which,the effect of the Map_LZO and Map_Snappy formats of distributed Map is the best,and when the record is more than2.0×107,compared with the time of direct query without compression at the query shuffle phase,the time has been greatly reduced,for about 31.6%.Thirdly,the speed Impala loads rail power supply monitoring big data is far faster than the relational database.What's more,Impala's query performance is also far better than the relational database and Hive,even for records of 10 million level,it can also complete query within hundred milliseconds,the query performance has been increased by about 3magnitudes,and has better interactivity.Therefore,this measure has certain practical significance for the quick query of railway power supply monitoring big data. |
PDF Full Text Request