Study On Online Analytical Processing And Data Mining In Geological Environmental Data Warehouse

Since the reform and opening up, China’s rapid economic development has made remarkable achievements, however, the continued growth of industrialization, urbanization and modernization of agriculture also led to geological disasters, groundwater over-exploitation and pollution, contaminated tailings and a series of serious geological environmental issues, which not only caused a growing hazard on people’s life, but also seriously affected the prospects for long-term development of China’s economic. Therefore, utilizing natural and ecological resources rationally, monitoring and preventing geological environment, reducing and compensating the adverse effects of the geological environment caused by social and economic development are becoming important works for maintaining the balance of nature, and promoting sustainable economic development.To this reason, China set up the national, provincial, ground (city) three level geological environmental monitoring and management system, invested a lot of manpower and material resource to carry out a wide range of geological environmental investigation, complete a comprehensive collection of basic geological environmental data, build the geological environmental monitoring network, and achieve the real-time collection and transmission of monitoring data. With the lapse of time, these geologic environmental survey data and monitoring data accumulated continuously has reached a certain scale, the existing geological environmental information processing systems are unable to effectively analyze and deeply mine these data.There are some problems:(1) the superior departments and subordinate departments, as well as the same level departments used different data storage platforms, data storage is dispersed and data format is not unified,it is difficult to realize the sharing of data between the superior and subordinate departments,as well as the same level departments; (2) During the process of forming data for reports delivered to superior departments, there are too many artificial intervention which are more likely to cause inefficient and inconsistent data between the superior and subordinate departments; (3) the amount of geological environmental data is becoming larger and larger, in analyzing geological environmental survey data and monitoring data, enormous inquirying, joining and aggregating operations consumed a large number of times, and the efficiency is too low to be tolerable; (4) Means of analysis and assessment of geological environmental surveying and monitoring data is not rich enough, existing methods did not take full advantage of the rich massive historical data to excavate data’s internal value, it is difficult to find the rules and modes of geological environmental evolution, and assess the impacts which various factors have made on geological environment.Therefore, it is necessary to extract the inconsistent massive geological environmental survey and monitoring historical data stored in dispersed departments, transform and integrate them into the unified storage space, provide a source for all-round analysis and and in-depth mining, expand and discover the new methods for analyzing and assessing survey and moniting data, find the evolution rules hidden behind the massive geological environmental objects, provide more scientific basis of decision-making for geological environmental management.These works have become the primary tasks of the new era of the information processing for geological environmental monitoring, preventing and treating.The content of this research is the geological environmental data warehouse construction project from the China geological environmental monitoring center, the goal of the project is to build a geological environmental data warehouse with higher degree of integration, achieve the data fusing, and realize the corresponding aided decision applications. This paper discussed the current domestic and international situation about analyzing and processing methods for geological data, the administrative organization structure and business processing of China’s geological environment management departments, distributing status of geological environmental historical investigating and monitoring data, and shortcomings of the existing geological environmental analyzing methods. On this basis, this paper proposed a complete solution framework of constructiing geological environmental data warehouse and decision support application systems. The whole framework can restructure the geological environmental data from operational databases, constructs geological environmental multidimensional data model, and achieves the processes of data cleaning, transforming and loading. At the same time, the framework also provides on-line analytical processing and data mining algrithms and tools, so as to provide support for various decisions, such as geological disaster forecasting and warning, groundwater monitoring and protectiing, mine geological assessing, resource carrying capacity assessing.The main research contents of this paper are:(1) This paper proposed and constructed a complete strcture of geological environmental data warehouse accompanying OLAP and data mining options, formed a framework of integratiing, analysising, mining, showing for geological environmental data. It covers all aspects of achieving data warehouse, such as data ETL(Extracting, Transforming and Loading), building OLAP multidimensional data model, real-time data analysising, and in-depth data mining. The strcture effectively improved the analyzing and assessing mechanism of geological environmental information, and developed a new approach for assistant decision-making of geological environmental management.(2) Based on analyzing the distributing situation and management processes of existing geological environmental data, the paper integrated the geological data, designed the system structure of geological environmental data warehouse, organized and stored the historical data of geological environment into different data marts, such as "geological disaster monitoring and management", "groundwater monitoring and protection"," assessment and management of mine geological environment", "geological heritage survey and protection", "geological environment and resource carrying capacity assessment".(3) Developed a set of performance optimization solutions for geological environmental data warehouse, using bitmap indexes, partition management, and asynchronous CDC-based incremental extraction strategy to effectively improve retrieva and analysis efficiency of the geological environmental data warehouse.(4) The paper also studied the method of constructing multidimensional analysis system of geological environmental data, designed and achieved a variety of geological environmental multidimensional data cube, and provided front-end systems to implement flexible online analytical processing.(5) The paper studied on data mining models based on the geological environmental data warehouse, discovered rules and patterns deeply hiding under the massive geological environmental data, designed many data mining algorithms, such as Support Vector Machine-based geological hazards time series forecasting algorithm, Neural Network-based groundwater quality assessment algorithm, Composite Index Analysis Method-based mine geological environmental assessment algorithm, Composite Index Analysis Method-based natural resources and environmental carrying capacity assessment algorithm.(6) The paper researched and designed a multi-platform solution of online analytical processing and data mining front-end system, achieved a flexible front-end system which can access the data of geological environmental data warehouse through web browser, smart phones, or tablet PCs, provided a variety of user interfaces for geological environmental data warehouse.The main features and innovative points are:(1) proposed and constructed a geological environmental data warehouse architecture, combining processes of data integration, data analysis, data mining, and result show as a whole, systematically changed the mechanism of geological environmental data analysis, and improved decision-making efficiency. (2) designed a performance optimization scheme of the geological environmental data warehouse which using bitmap indexes, partitions, and incremental extract strategy to effectively improve the operating efficiency of the data warehouse. (3) provided cross platform, flexible front-end system interface, so that obtained more diverse way for usingthe data from the data warehouse.Lacks of paper are:(1) online analytical processing platform Mondrian although has good portability, but there are still some defects in the memory allocation and processing, and it is difficult to adjust the internal implementation details. (2) Due to the limited study time, as well as unfinished data collection progress in many fields, no further data mining algorithm was validated and compared in the geological environmental data warehouse.In short, the emergence of the geological environmental data warehouse, online analytical processing and data mining systems is the inevitable change of geological environmental data processing methods, it is bound to improve the scientificity and efficiency of geological environmental management and decision-making.