Researsh On Distributed Compression Storage And Fault Tolerance Of Monitoring Information Flow Of Power Supply System Of Electrified Railway Based On Hadoop

The scale and the capacity of modern power system expand constantly, the interactionsbetween the energy and the information flow construct the basis for the intelligent dispatchingand monitoring system, in order to meet the coordination and communication requirements ofthe dispatching and monitoring system, more and more new kinds of electronic and electricalequipments are integrated into the system, leading that the data of the dispatching andmonitoring system increases in a geometry form, and data type and structure becomescomplex and diverse. Typical example such as one turbine SCADA system of a wind powerstation, the related produced data amount of the system reaches about2490GB per month,which will reach more than20TB in a year[1,2]. Now the existing data management systemsgenerally use the traditional disk array to store and manage the data, the corresponding datamanagement software uses the relational database processing method, so that the centralizeddata processing in single server requires a higher performance, it also has weakness such asrelatively poor scalability and relatively poor reliability, altrough it can meet the general dataprocessing applications, it is difficult to deal with the massive big power data, big dataprocessed centrally by using the traditional relational data management system has been farfrom the requirements of the efficient storage, effective processing, data sharing andautomatic control of the future dispatching and monitoring system. How to process the bigdata of the current dispatching and monitoring system effectively is still one of the challengesand problems.Cloud computing is introduced as as one kind of solution to solve the big data problemsby academic researchers in power field, Hadoop cloud computing is researched popularly,which is an open source cloud computing platform, breakthroughs the traditional centralizeddata processing model, it has powerful distributed parallel data processing performance, candeal with the big data effectively, has good performance of high reliability, high efficiency,high expansion and other outstanding advantages, providing a new processing method for thebig data produced by the dispatching and monitoring system.Some special industrial power distribution system such as the power supply system ofthe electrified railway, not only has characteristics of the general power distribution system,but also has some special characteristics of its safety not only relation with the power supply,but also relation with the transport dispatching, which requires a higer reliability performance.Combined with the actual data access demands with the Hadoop cloud computing platform, dispatching and monitoring system of the electrified railway based on the Hadoop isconstructed, then the distributed dispatching and monitoring data warehouse based on Hiveand the distributed dispatching and monitoring database based on HBase are constructedbased on the Hadoop platform in the paper. Thus, take the big dispatching and monitoringmeasurement data as the data object, to research on the distributed lossless data compressionstorage, experiamental results show that, RCFile has a better query performance than that ofTextFile and SequenceFile; and query and compression performance are improved afterDeflate, Gzip, Lzo compression; and verifies that HBase has a good reading and writingperformance; and integrate the distributed database HBase into Hive, realizing theinformation interoperation between them; and verifies that dispatching and monitoring systembased on the Hadoop has a good fault tolerant performance. The research conclusions have asignificant theoretical and practical value for the big data processing of the dispatching andmonitoring system.