Research On JSON-based Data Exchange Between Heterogeneous Data Sources

With the advent of big data era, the traditional data platform solely consisting of relational databases cannot adapt to the efficient management of large volumes of unstructured data, while a variety of heterogeneous data sources join the data platform, the management performance of Internet heterogeneous data is greatly improved. However, in the interaction of the data from heterogeneous data sources, application developers need to write different source code for different data sources. The number of source code is large and the structure is complex. It is difficult to maintain and extend, resulting a big burden to developers.In order to improve the efficiency of data exchange code development and the maintainability of heterogeneous data sources, this thesis studies data exchange model of heterogeneous data sources based on JSON, in the form of database middleware in the heterogeneous databases and acting as intermediate interface between the client logic modules for developers to use. This thesis presents the research work in four parts:Firstly, for the new requirements of big data storage put forward by the Internet applications, it studies the existing multi-database interaction cases and presents JSON database middleware model combined with the emerging NoSQL databases.Secondly, it analyses the internal data formats used in MySQL, MongoDB and Redis, as well as JSON data formats. Through the analysis of these three kinds of database format for storing data, it finds out the most suitable databases to deal with which given data, and provides references for seeking solutions to handle complex data using multiple databases.Thirdly, it implements a unified API using the Java language. The interface includes the data transfer functions of MySQL, MongoDB and Redis with JSON. Developers can invoke the specified interface function to accomplish the exchange function.Finally, the unified data exchange API has been tested its functionality, and used in the actual project- risk management platform for cranes. Comparing to native code, the middleware model designed in this thesis has achieved data transfer functions and has a good effect of encapsulation and extensibility.