Research On Database Migration And Schema Conversion In Application In Commodity Price Comparison System

With the rapid development of Internet, our society has come into information age of the high-speed development. All kinds of Internet applications are experiencing explosive growth, which produce huge amounts of data. Since the1970s, relational database management systems (RDBMS) have dominated the data landscape. But as businesses collect, store and process more and more data, relational databases are harder and harder to scale and becoming poorer and poorer on query performance. In some high concurrency network applications, the database layer suffering heavy load demands higher performance and expansion. Facing these demands, the traditional relational databases are becoming more and more overwhelmed. With good expansibility and high performance in dealing with big data, The so-called NoSQL databases have been leading the charge and creating all these new persistence choices gradually.HBase is a widely commercialize applied NoSQL database in recent years. It is a database that provides real-time, random read and write access to tables meant to store billions of rows and millions of columns. It is designed to run on a cluster of commodity servers and to automatically scale as more servers are added, while retaining the same performance. Having been successfully used to build real-time systems by the well-known Internet companies such as Facebook, it becomes one of the most popular NoSQL databases at the moment. In view of the practical application of large amount of data migration and transformation requirements, this paper discussed the database migration methods from relational database to HBase under the background of vast amount of data.In this thesis, the author decide to design and implement the database migration systems based on XML middleware solutions after studying the basic theory of database and some classical data migration method and strategy. The main works are shown as follows:1) Aiming at the present problems of database migration from RDBMS to HBase, this paper proposes a kind of effective database migration and data schema transformation method from RDBMS to HBase based on XML.2) Introduce the design and implementation of the database migration middleware in detail.3) By successfully applied in the commodity price comparison system, the database migration solution successfully migrates the historical data from RDBMS to HBase.