Seismic Achievement Data ETL Platform Architecture Design And Software System Implementation

Seismic achievement data is directly associated with some economic benefits like well placement success rate, oil discovery,and oil recovery, and that it is the key to find oil and gas resources. Now oil workers have collected more complete static and dynamic data information and accumulated a large number of precious resources for the oilfield. However, the data is scattered throughout the oilfield and heterogeneous, and that omissions error and other issues often appeared when the data was saved and loaded. And at the same time they can’t achieve a unified querying, loading, and data synchronization refresh between the main library and the user’s work environment.This system puts forward solutions to solve these problems of software development.This paper took Daqing oilfield Exploration and Development Research Institute’s project named "The development and upgrading of seismic data ETL(Extract- Transform- Load) module" as the project background, and the seismic achievement data ETL system was developed according to the actual situation and the needs of the oilfield. System architecture and layered design were based on WCF’s oriented service and the Entity Framework entities architecture technology; Using WCF technology makes system deployment more flexible, and that the system can respond to d emand change more quickly; Under the premise of same EF storage layer, we used this layer to block the influence of the target layer of data source; Combining with the actual needs of oilfield exploration and development, data characteristics and application fields, analyzing the correlation between the source data in the storage model, using XML universal description, establishing universal description framework model, and according to the corresponding relationship between source data and target data, we defined the one-to-one, one-to-many and many-to- many model mapping relationship; The ETL process established data migration scheme; According to the scheme to realize data extraction and transformation algorithm, the algorithm defined a two-dimensional table structure, transformation rules and the transformation function used for temporary storage, and that the algorithm used the method of first order abounded again to complete the transition between XML document and trees, and that the data in documents was stored in the two-dimensional table; The realization of the data management quality control was in entity and controlled the relevant fields of the entities including integrity, consistency, and three kinds of data detection method based on business rules; And we used small batch, continuous and main-table loading strategy for data loading.Finally the system realized functions such as seismic achievement data correlation analyzing, querying,sharing, uniform loading and quality control, etc.Currently, the seismic achievement data ETL system has been put into using and the operation was stable. The system has solved the seismic achievement data storage and dispersion, fast search difficulty, the low level of information association and sharing and other problems, and that it has improved the level of data integration, application and sharing. And it has also met the needs of users and improved the users’ work efficiency.