Research And Application Of Integration Method For Multi-source Heterogeneous Data

With the development of big data,mass data exist in different information systems.Due to the distinct difference of application purposes and data storage specifications,data in various systems are heterogeneous and isolated.The multi-source heterogeneity among the data makes it impossible to achieve unified management and sharing.It is easy to produce information island,technology island,resource island and other problems.And it becomes difficult to carry out joint analysis of realm data items.In order to solve the current problems and give full play to the value of data,it is necessary to integrate multi-source heterogeneous data.Data integration is the key technology to realize information sharing and build unified access to data.The paper focuses on the data integration among multi-source information systems in universities.In the current business scenario,data sources are designed by different organizations with different models or specifications.Due to the limitation of technology and human factors,the storage modes and standards of different data sources are various.Then problems such as pattern heterogeneity and semantic heterogeneity begin to arise.How to improve the utilization of heterogeneous data becomes the problem to be solved.In order to realize the associative access between heterogeneous data sources,a data integration scheme is proposed to integrate the heterogeneous data sources effectively.In order to overcome the problems of low query efficiency and low degree of automation of traditional data integration scheme,this paper introduces semantic network to integrate data based on ontology technology.In order to improve the expression ability of ontology,the database metadata information is introduced to carry out association analysis based on BP neural network algorithm.The research contents are as follows:(1)Integrate data based on ontology technology.First,build local ontology based on multi-source database.For local ontology,adopt various similarity algorithms to match local ontology and solve semantic conflicts between different ontology concepts and attributes.In the convergence operation of similarity matrix obtained by a variety of similar algorithms,the influence of different matrix data characteristics on the results is considered,and the matrix fusion is carried out by means of dynamic adaptive weights to ensure the objectivity of the results.Finally,the mapping relationship between concepts is selected by the stable marriage algorithm.(2)In order to make full use of the database metadata information to improve the matching ability of the ontology,BP neural network algorithm was introduced to carry out association analysis on the ontology matching process,and the self-learning ability of the neural network was used to carry out attribute matching.Finally,the mapping relationship between local ontology concepts is calculated by the combination of ontology similarity calculation and BP neural network,and the domain global ontology is constructed by the mapping relationship.By transforming the query of multi-source information system into the query of global ontology,the integration of multi-source heterogeneous data is completed.(3)The data integration method based on ontology and neural network proposed in this paper is experimented on public data set and real scene data set respectively.The experimental results show that the proposed algorithm has good performance in data integration.Finally,combining with the actual application scenarios,the algorithm in this paper is applied to the heterogeneous data integration platform to provide unified data query view for data users and realize the joint access between multi-source heterogeneous data.