Research And Implementation Of A Flight-Oriented Training Data Centre

With the rapid development of information technology,the civil aviation industry has also entered the era of information technology and digitalization.However,due to historical factors,civil aviation now has a wide variety of data and inconsistent informationization,and in the process of civil aviation informationization development,there is a problem that various civil aviation systems are unable to collaborate with each other and the same civil aviation data cannot be shared.Based on the concept of building a "Big Data+Flight Training" infrastructure platform proposed by the Civil Aviation Administration in recent years,this paper designs and builds a data centre for flight training with flight parameters as the support,integrates heterogeneous data from multiple sources and precipitates data knowledge through data modelling and analysis,eliminates data barriers,and realises flight training data in the form of data interfaces.The data interface is used to realize the common data and data services for flight training.The details are as follows1.To design the data posting layer for heterogeneous data from multiple sources in civil aviation,the flight parameters from the GARMIN1000 system in the Cessnal 172R training aircraft,the flight parameters from the CESSNA CJ1 simulator,the pilot training test score levels,flight training logs and other data are collected as data samples for the middle platform,and the file data are simply cleaned and converted using ETL technology.The video data is then pre-processed by Matlab into still images and aggregated using Flume for multi-node data collection.2.The data governance layer is built for the flight training data fusion mechanism,specifically implementing the flight training data pre-processing module,the data warehouse detailed implementation module,the training log matching logic module and the data centre service module.In terms of technology selection,HDFS+Hive is used to build the data warehouse hierarchical model and data analysis;the matching logic of flight parameters and flight training logs is implemented by means of multi-segment matching;Kafka is used to monitor the data service;DBeaver is used to perform view queries on Hive and MySQL;DataEase is used to design the data visualisation big screen.The Lambda architecture,which can handle batch data and incremental data separately with a certain degree of robustness and fault tolerance,was chosen to provide a more complete representation of the governance capabilities of the data centre.3.The data service layer was designed to meet the user’s application requirements,and the built data centre for flight training eventually implemented the service functions of the centre system,data reports,data screens and view queries.Through environmental testing and system testing of the data centre,the cluster deployment server nodes were planned to test and verify the hardware and software environment,memory occupation rate,system performance,system compatibility and centre system functions,so that it has the service capability of a mainstream platform and achieves the purpose of fusing training data from multiple sources and realising the value of data sharing.