Metadata Management System For Transport And Logistics Big Data Process

In the field of transport and logistics, establishing a unified, efficient data management way is an urgent need of reducing operating costs, improving data quality and standardizing logistics management. The paper is on the background of transport and logistics cloud platform project of a province, which contains a series of transport and logistics big data process requirements like integrating data from logistics enterprises, application developers and data center as well as managing them. So far, in data integration and management, some problems exist: 1) no mapping is established between business metadata and technical metadata thus no unified management is taken among heterogeneous enterprise data sources; 2) business metadata and technical metadata are not combined to guide data integration; 3) as it relates to safety and privacy issues, the access to enterprise private data sources in data integration is difficult.To solve these problems, we first study deeply in metadata standard SDMX information model, RDF data cube vocabulary, data integration and other related technologies; and propose a business metadata model for the transport and logistics field; then we establish the mapping between business metadata and technical metadata; and with the combination of an ETL tool, we design and implement a metadata management system for transport and logistics data process. Test and application indicates that our system is feasible and effective.Compared with other similar systems, our work has following features:1) The paper combines SDMX information model and RDF data cube vocabulary, proposes a business metadata model for the transport and logistics field and build models for terminology and business entities in the field of transport and logistics as well as establishes and maintains the mapping between business metadata and technical metadata.2) The paper proposes a transport and logistics data integration approach based on metadata by using business metadata in data integration. When users develop data integration tasks, the mapping between business metadata and technical metadata can help to automatically create the matching between source and target integrating fields, which simplified the development process.3) The paper designs and implements a metadata management system, including a metadata management tool and a data integration tool. The metadata management tool provides technical metadata harvesting, metadata management, metadata mapping, agent management and system management functions while the data integration tool provides data integration tasks developing and data integration executing functions. Application result shows that the system enhances the data management capabilities and improves the data integration efficiency.