Research On Data Standard And Data Synchronization Technology In University Data Fusion

In recent years,with the rapid development of information technology and the continuous deepening of the informatization construction of universities,information management systems customized by the various business departments of universities are gradually increasing,resulting in various types of data which is in large number,scattered,and with complex structure,thus greatly affecting the embodiment of the value of the data.How to transform the isolated and scattered heterogeneous data into highquality and effective shared data has attracted wide attention from researchers.Data fusion can effectively improve data utilization and ensure data consistency by dynamically extracting and transforming various business data,and finally loading it into a large-scale unified modeling database.In order to better realize data fusion,it is not only necessary to formulate a unified data standard based on the actual situation of universities,but also to dynamically synchronize the constantly changing data.Therefore,this article will design and implement data standards and data synchronization.The main content is as follows:(1)In view of the inconsistency of data and the lack of data standards in the data fusion process of various business systems,this thesis fully investigated the current situation and needs of informatization of a university,analyzed the basic data of the university in combination with relevant standards,and formulated a set of relatively comprehensive data standards.(2)In this thesis,the data lake is introduced as an intermediate database in the overall architecture of data fusion,and the data synchronization process is split into two steps to achieve.The first step is to synchronize business data to the data lake.The key technology of this step is change data capture.For this reason,a hybrid change data capture method of control table method,timestamp method or log method is proposed.According to the openness level and structural characteristics of the business system,specific methods are selected.The second step is to synchronize the data from the data lake to the data center.The key technology of this process is to realize data conversion by referring to the data standard file.The data conversion is realized by introducing the data conversion code of Kettle,an open source tool of ETL.The process will be completed in the data lake and data center,reducing the impact on the database performance of the source business system.(3)This thesis implements the designed data synchronization scheme systematically,and verifies the effect of data fusion and data synchronization through an information query based on the data center.The test results show that the effects of fusion and synchronization have satisfied the expectations of this thesis.