Design And Implementation Of The Data Migration Tool For GBase

As one of the important domestic software that the government supports, domestic database has been applied more and more in the domain of e-government, information, security, education, people and so on. The data migration tool, as a part of DBMS, will migrate data if the old DBMS and new one switches. Inheriting the precious data and making sure the new system run favorably is a important basis on judging a data migration tool.Most database manufacturers have their own data migration tool. Some of them are very excellent product, for example, the DTS for Microsoft SQLServer, Oracle Warehouse Builder for Oracle and the third party's DataStage.Data migration tool for GBase is designed on referring the magistral migration tools with the principle of the essential requirement for leading users, and developed on the basis of JDBC interface. In the aspect of integrity, the tool supports not only the table structure and data, but also the constraint such as primary and foreign key and index. For the database type, it supports migration among seven kinds of magistral databases including Oracle and GBase. Additionally, Data migration tool for GBase allows to migrate between database and three kinds of file, including"TXT","EXCEL"and"XML".Data migration tool for GBase achieves the migration task by three function centers. They are the storing center, the syntax composing center and the controlling center. By simulating the storage structure for database, the storing center fetches and stores four-grade metadata including database, schema, table and column. The syntax composing center will analyse and compose the SQL syntax consisting of column, table, primary key, foreign key, and index and so on. In addition, the communication between GUI and background and the alternation among modules in background is achieved by the controlling center.As a whole, the migration process consists of fetching data, transforming data type and loading data. Transforming data type is the most crucial step. It will be fateful for the result and quality of migration whether the mapping relation between data types for different database is accurate and suitable. In data migration tool for GBase, we design a model linked up with JDBC data type to transform data type among databases. In the model, the full map between JDBC data type and database data type is a basal unit, known as a JDBC data type map. Therefore, if we want to transform data type from the"source"to"target"database, what we need to do is matching twice with the JDBC data type map for"source"and"target"database. With the map about data type of"source"and JDBC and"target", the data type will be transformed by fetching and loading data with the java data type corresponding to theJDBC data type in the course of migration. Based on analysing and practicing kinds of database system, the paper summarizes detailedly the relation between all kinds of database data type and SQL2003 data type, which provides the foundation to customize the JDBC data type map for every database the tool supports.The migration sequence for objects is controlled by the fore-and-aft migration options. It is that the structure of all tables is migrated firstly, then data, and last is the constraint. The reason is to avoid the problem brought for the reference among tables such as foreign key and index. And for the migration of constraint, the tool will migrate indexes firstly and then foreign keys, because a foreign key may be not only created on a primary key but also a unique index.One marked feature of the data migration tool for GBase is the balance of automatization and flexibility. In the aspect of flexibility, the migration item is optional so that user enable to choose one or some items among table structure, data, primary and foreign key and index; the tool supports migration for seven kinds of magistral database, meanwhile, it allow user to load new JDBC driver and migrate for the new database; before migration task works, the data type map and table structure syntax are editable, which leave the chance for user to alter when some illegimate data type or SQL syntax appears. In face of automatization, the tool offers JDBC data type map for the seven kinds of magistral database including Oracle, which avoid that matching data type in the process of migration and enhance the efficiency; additionally, the function of matching data type map automatically is convenient for users to migrate the database for the new database they load.Performance is anther important guideline for users to pay close attention. The paper introduces some main strategy for performance optimization in data migration tool for GBase. Concurrency for different migration tasks implemented by multithreading, which has not been provided in other homogeneous products, enable users to execute more than one migration task at the same time. Some optimization method in the process of JDBC I/O improves the efficiency. For example, fetching no less than one data record for query, committing more than one"INSERT"SQL by"Batch", etc. After the performance test with the data in TCP-W testing, the paper shows the comparison result of migration rate for some magistral databases. Data migration tool for GBase is much near to DTS for SQLServer, and almost equal to the main domestic databases.At present, the data migration tool for GBase is applied in important projects of some institutions and companies including International Department Central Committee of CPC, GEER in shanghai and one graduate school of ministry of information industry. And the tool wins the first place in the evaluating of domestic database organized by CSIP. However, there aresome problem to resolve in the future, for example, researching the data types which are not supported now, analysing the migration of partition, investigating and surveying the migration of stored procedure among isomerous databases.The related standards and specifications on database, a newly emerging technology, are being established and completed. Data migration, as the cornerstone of database system, deserves much applied research and practice.