Design And Implementation Of A Big Data Organization And Storage System For Dynamic Monitoring Of Soil And Water Conservation

Dynamic monitoring of soil and water conservation refers to the investigation,observation and analysis of the occurrence,development,hazards of soil and water loss and the effect of soil and water conservation,so as to obtain dynamic detection data Finally,it provides guidance basis for the formulation of relevant ecological environment management measures.However,the utilization rate of domestic soil and water conservation data resources is not high,and the lack of a centralized monitoring data platform,unified data format and insufficient interdepartmental collaboration make it difficult to share data.In view of the above problems,this paper analyzes the characteristics and data flow of massive data generated in the process of dynamic monitoring of soil and water conservation,designs a storage system for big data of water conservation.By using database technology and distributed storage technology,technical specifications of data organization,storage,access and other links are established to realize efficient organization and storage of massive dynamic monitoring data of soil and water conservation and give full play to the value of the data.It proves by practical application that the system has availability and flexibility,and can provide better data support for other business subsystems of the platform.Research contents and achievements of this paper are as follows:1.A unified organization and management scheme of spatio-temporal big data is proposed.Based on the existing spatio-temporal data organization modeling results,a version control scheme is proposed to describe the computing tasks and background files together,and an extensible file storage directory structure is established for dynamic monitoring of unstructured data.2.A storage scheme for dynamic monitoring of massive multi-source heterogeneous data is proposed.It supports a variety of data storage modes,such as vector raster and other unstructured files,structured data stored in table form,etc.3.On the basis of data organization and storage scheme design,a hybrid storage system aiming at mass data of national soil and water conservation dynamic monitoring is constructed to realize distributed storage and centralized management of soil and water conservation dynamic monitoring data and provide unified data access services to enhance data flow.