Real Time Data Analysis System Of ISCS Based On Hadoop

Due to the low integration and closeness of traditional subway stations,as well as the large amount of data generated in the driving,diverse data structure and real-time data,the data storage and processing system can not fully meet the needs.In this paper,we build a server cluster to collect management data using cloud platform technology on the existing traditional station network.The specific method is to add Hadoop distributed system cluster,extract all data information from the integrated monitoring central real-time server of metro operation,and store the massive data in the non structural database HBase through the pre signal processor.The advantage of this is that it can save the original real-time data of the main line operation line to the greatest extent,analyze and count the operation characteristics of the main line through these data,form the passenger flow distribution and power consumption that can directly reflect the operation characteristics,and give the operation suggestions.This topic the main research results obtained as follows:1.The interface design of data acquisition,data storage and interactive query function based on HBase.It includes obtaining massive real-time running data from running lines,network of front-end computer and forwarding design.The master node divides data into multiple slave nodes according to specific rules(communication,power,electromechanical,etc.),and tracks the actual storage location of data.The master node of Map Reduce is responsible for assigning calculation tasks to the slave node,and each slave node is responsible for specific data storage and calculation.2.Improved application of distributed SVM classification algorithm.It includes the improvement of data classification algorithm and the realization of multi-node parallel algorithm.The classification algorithm is based on the realization of Metro data optimization algorithm.Through this algorithm,the data can be allocated to the slave node according to certain classification rules,and parallel calculation,analysis and statistics can be carried out.3.An experimental environment for testing the accuracy of training set and the efficiency of distributed system is designed and implemented.It includes the establishment of experimental environment and data preparation,the comparison test of system database storage and query efficiency,the parallel efficiency test of real-time data analysis of metro,and the parallel comparison test of data from multiple stations to the whole line.4.The real-time data of subway operation are analyzed and processed,and the operation curves of each specialty are drawn.This paper studies AFC data curve and electric energy data curve,draws real-time curve of station and the whole line,and analyzes and forecasts the curve fluctuation law.The work of this paper provides a new cloud scheme for the expansion of business functions of traditional stations,which can also be used as a part of the functions of intelligent stations,providing valuable reference for the realization and transformation of intelligent stations in the future.Previous studies have stopped at a certain line or even a station.In this paper,through the comprehensive monitoring of the data collection of various disciplines and stations,so that the original unusable data has been used,the operation trend of each discipline can also be analyzed and the relevance of them can be obtained.