Research On Multi-source Heterogeneous Internet Of Things Big Data Integration Technology

Multi-source heterogeneous sensor data in smart cities cannot be directly compared and exchanged and shared due to different data models and data standards,and queries between heterogeneous data sources are also difficult to achieve,and there is a problem of low sensor data utilization.Therefore,this paper will focus on the data integration of multi-source heterogeneous sensors.Based on the Semantic Web technology and related ontology,this paper proposes a semantic annotation algorithm to add a semantic layer to the sensor data to achieve automatic computer integration;complete the data association between multiple heterogeneous data sources,which can mine hidden connections of data and become a smart city.Different application scenarios provide multi-source heterogeneous data single-point real-time query and historical data statistics.Semantic integration of smart city sensor data is a novel and practical research topic.The main contents of this article are as follows:(1)A distributed data integration system based on semantics is proposed.First,it summarizes the development process of smart city architecture and Io T data integration systems at home and abroad;it is proposed to add a semantic layer to the existing data integration system to make up for the shortcomings in the existing sensor data integration system,and enhance the generality of the existing integration system.Sex and data association capabilities.The specific design adds the semantic layer of the distributed integrated Internet of Things data system structure,and designs different sensor data processing and data calculation schemes for different application scenarios in the integrated system.(2)A general framework for semantic integration of multi-source heterogeneous data based on multiple ontologies is proposed.Proposed semantic annotation algorithms based on SSN,SAO,GEO and other ontologies;and realized near real-time incremental real-time sensor data semantic annotation and higher performance historical archive data semantic annotation;made up for the lack of incremental existing integration framework.Disadvantages such as integration and unrealized unified query interface of historical data and real-time data.The framework provides a unified data model for the annotated sensor data,so that the computer can understand the integrated sensor data and realize automatic data integration.According to the comparison of experimental results,it is concluded that the integration framework proposed in this paper can effectively integrate multi-source heterogeneous data.Compared with other frameworks,the query performance is better and the query response time of the sighted framework is at least reduced by half.(3)Research and implement data association between multiple heterogeneous data sources.Experimental comparison of multi-source heterogeneous sensor data in 4areas of smart city was carried out to achieve multiple data associations between data sources.In 6 application scenarios,the data association was used to perform single-point query and real-time multi-source heterogeneous sensor data.Update incremental data queries and statistics for long-period historical data.Experimental results show that data association helps to query hidden connections between multiple data sources and the data query performance is good.For long-period data statistics queries can be within 0.1 seconds return the result.