Research And Application Of Hybrid Architecture Storage And Retrieval System For River And Lake Space Data

The management and protection of rivers and lakes are effective measures to promote the harmonious coexistence of human and nature,and accelerate the construction of ecological civilization.In recent years,with the continuous development of satellite remote sensing,Internet of Things,intelligent sensor monitoring and other technologies,the volume of river and lake spatial data has increased explosively,and its data types are diverse,formats are complex,structure is complex,time and space are interlaced.Effective storage,management and retrieval of large spatial data in rivers and lakes are facing enormous challenges.At the same time,higher requirements are put forward for data concurrent access support capability and security reliability.Furthermore,the national project on river and lake management and protection has risen to the level of national strategy.The protection of the Yangtze River and the governor system of rivers and lakes all involve the spatial control of rivers and lakes in river basins.Therefore,it is urgent to carry out research on intelligent management and control of large spatial data in rivers and lakes.Taking the storage and retrieval of large spatial data of rivers and lakes as a breakthrough point,this paper develops a hybrid architecture storage and retrieval system for large spatial data of rivers and lakes,which solves the problems of efficient storage and retrieval of massive multi-source heterogeneous data and concurrent access support.A unified interface specification for multi-source heterogeneous data is established to implement a new mechanism of orderly sharing,moderately open,safe and reliable large data sharing in river and lake space.It has great theoretical research value and engineering application value in promoting the great protection of the Yangtze River,promoting the comprehensive effect of the governor system of rivers and lakes,and implementing the fine management of rivers and lakes.The main contents and innovative achievements of this paper are as follows:(1)In the aspect of structured data storage,combined with the technical specifications of water resources and geographic information related countries and industries,the existing water-related management system is migrated and integrated by using the "one-source-onedatabase" model,avoiding repeated collection and storage of observation data,improving data consistency and facilitating data sharing among various subsystems.(2)HBase and MongoDB are used to implement raster image data storage and retrieval.HBase-based tile pyramid storage is used to implement fast visual display and query of data.Metadata model based on MongoDB document model is used to simplify metadata description.At the same time,the Elasticsearch search engine is used to build metadata fulltext index,which provides multiple matching methods such as exact matching,fuzzy search,and range query.(3)Establish spatial vector feature data storage model based on GeoJSON and MongoDB,build spatial index,design auxiliary index to accelerate data query and filtering;design sharding strategy in sharded replication cluster,balance the contradiction between data distribution and query efficiency.(4)Aiming at the need of massive data management and control in river and lake space,a hybrid architecture system based on large data technology is constructed to implement the management and control of multi-source heterogeneous data in river and lake space,provide a unified data transmission and retrieval interface,and strengthen the interconnection and interoperability of data.At the same time,the system provides concurrent access support to ensure the security and reliability of data.