Research And Implementation Of Hadoop And Rdbms Mashup Data Management

With automation and accelerate the speed of data generation, the amount of data that needs to deal with rapid expansion. Relational data management technology for in-depth analysis of the data capacity is insufficient, while Hadoop and MapReduce technique lacking in terms of real time and it cannot completely replace the relational database. Therefore, there are many RDBMS and Hadoop integration technology solutions for data storage and processing.This paper focuses on some of the traditional RDBMS and Hadoop integration solutions in data exchange and management while with parallel data mining project as background, and proposed Hadoop and RDBMS data exchange and management programs. The main work includes:(1) Develop HDFS data exchange platform. Using this platform, data can be shared between Hadoop and traditional data management systems, and used large data processing technology to mining deeper value.(2) Study and research Hadoop Distributed File System (HDFS), and provide a more abstract HDFS file management, user-friendly and manage files.(3) Hadoop data sharing between different data processing tools. Utilization (2) technology, HDFS provide external access interface more friendly, and direct support for a variety of Hadoop-based data processing tools, such as MapReduce, Hive and Pig.