Study Of Management And Optimization Methods For Massive Marine And Atmosphere Data Based On WEB-MAGIS

WEB-MAGIS is a Geographic Information System (GIS) software platform developed specially for marine and atmospheric fields. It integrates seamlessly three functions including data service, spatial-temporal analysis and visualization service, which provides customers with efficient, flexible and online services. Marine and atmospheric data have many characteristics such as varietal sources, different forms, long time series, massive volume and so on, and provide reliable data sources for the functions of WEB-MAGIS as the basis of the platform. In this thesis, it describes how to manage the marine atmospheric preprocessed data by ORACLE database. It also displays several optimization schemes for management of marine and atmospheric data to optimize data storage and improve the efficiency of data inquiry. The main research contents and results are as the following:1. Research on data preprocessing methodsBecause the data in different forms are obtained from a variety of sources, it needs to preprocess raw data for another processing procedure at first. In this thesis, the process of data preprocessing includes data loading, converting data format, processing of null, invalid or error value, which realizes the data quality control.2. Research on the management of marine and atmospheric data and the basic optimization based on distributed databaseBased on ORACLE database, the marine and atmospheric data, correlative dataset and metadata of data files are managed by traditional relational structure. The database is optimized by the traditional technology of memory optimization, query language optimization and partition to improve query efficiency.3. Research on management marine and atmospheric data by distributed databaseQuery efficiency and storage space are the most two important factors to consider because marine and atmospheric data application is usually query-oriented. Bases on these demands, researches on the optimized storage of oceanographic data change the traditional relational storage mode. The processed data are stored in different database servers. A method of distributed data management is achieved. And the methods of data manipulation are introduced.4. Research on marine and atmospheric data management by data warehouse According to the fact of the characteristics of marine and atmospheric data which include long temporal series and poor operability are in accordance with the characteristics of data warehouse which are subject-oriented, integrated, relatively stable and reflecting historical changes, this thesis proves the suitability of data warehouse in the solution of marine and atmospheric data management, designs the structure of subject-oriented data warehouse, and displays its specific implementation procedure.In this thesis, on the basis of research results above, the integration and management of marine and atmospheric data are achieved, the data with the proposed schemes are optimized and correlative results contrasted. At the same time, the marine and atmospheric data are applied to WEB-MAGIS platform, providing service to users as data source of this platform.