Research Of Log Analysis Method For Incremental Acquisition In Data Integration Application

The first step work of Data integration to effectively extract from data sources, take efficiency and data transmission quantity into consideration, the best method is to timely detect incremental data. The classification of information sources mainly contains databases and flat files. Due to feature of Information sources and the needs of business level, you can choose most suitable one from a variety of incremental acquisition method.This paper focuses on the log analysis method, which is dependent on the database system with a log component, I have implemented a log analysis method on the oracle database system. Also summarizes the theoretical basis and use notes of logminer tool from existing data, which are based on my own practice. First described two major problems:the effectiveness of the log extraction and net effect of incremetal data, especially this piece of log extraction is a key basis for the method, according to two different analytical mode, and these three cases:the initialization operation, normal operation, interruption restart,this paper illustrate the likely problems and how to resolve the idea, the main purpose is to reliably and effectively extract specific IDU operation in accordance with specific object. The second problem describe the theoretical models of net effect processing according to the transaction record's interchangeability and simplified rules, and on this basis, dealing with different situations and resolve difficult problems; Secondly, make a brief introduction of multi-source analysis, composite transaction Concepts, and describe a number of existing difficulties, for example, the control of composite transactions, multi-source operation, a large transaction processing, etc.