Research And Implementation Of Key Technology On Multi-source Data Preprocessing For EMU Operation And Maintenance

With the continuous extension of the high-speed railway network in China,the attention to the efficiency and quality of the EMU is also increasing.In order to analyze more comprehensively and accurately,we need to combine multiple data sources and get more comprehensive and richer data about the operation and maintenance of EMU from different business systems to support complex analysis.However,there are many problems,such as large data,heterogeneous,multidimensional,multi scale,unsynchronized,incomplete and so on.So before analyzing these multi-source data,how to preprocess multi-source data becomes a difficult problem.Therefore,in this paper,in view of the analysis of the operation and maintenance of EMU,the various problems in the multi-source data,the related technology of multi-source data preprocessing and the analysis of EMU operation and maintenance are studied.A complete set of multi-source data preprocessing system for operation and maintenance of EMU is put forward,and the data distributed in different data sources are processed into clear,complete and meaningful data.Thus,it lays a solid foundation for the related research and analysis of EMU operation and maintenance.The main work of this paper includes the following three points:(1)Study the related theories and methods of data preprocessing,including two scenarios:regular data preprocessing and multi-source data preprocessing.This way the main tasks of multi-source data preprocessing are summarized.Based on this,the problems of multi-source data and multi-source data preprocessing in EMU operation and maintenance analysis are analyzed.(2)Aiming at the problems existing in the multi-source data pre-processing of EMU Operation and maintenance,we put forward specific solutions,and put forward a complete set of multi-source data preprocessing system for operation and maintenance of EMU.The FCSPM(Fast Convergence Sequence Pattern Mining)algorithm is proposed to solve unsynchronized and incomplete problems that are difficult to handle.Through experimental comparison,it is proved that the algorithm has better processing effect than other similar algorithms.(3)Finally,according to the status of the existing EMU information system,without changing the existing information system,we realize the online preprocessing function of vehicle monitoring data based on Kafka,and achieved good results.