Design And Implementation Of The ORM Framework Supporting Database Sharding And Read/Write Seperating

With the rapid development of the Internet, more companies adv ertising on Baidu search engine to achieve cash flow. So advertising system's data showing explosive growth, a single database server has no longer enough to store all of the data, so the system sharding the databases according to a set of rules, and provides masters and slaves to separate database read and write to improve read performance. So the system needs a unified module to interact with all of the databases. Although there is many open-source Object Relational Mapping(ORM) framework, these frameworks are universal so they can not fit the business and databases very good. The current ORM framework in the system runs with poor performance, and wastes a lot of the cache servers' memory.In order to interact with the databases and cache servers beeter, and save the memory of the cache servers', a requirement of an ORM framework supporting database sharding and read/write separating is raised. The framework use JAVA language to develop, and interact with MYSQL and REDIS cache server. It is divided into six modules: configuration management, data source management, mapping management, cache management, data access and data reporting. The configuration management finished the definition and load of the configuration file of databases' and cache servers'. The data source management finished how the database and cache configuration is parsed and regis tered as corresponding data source object, and how to manage and find data source centrally. The mapping management finished how the JAVA object is mapped to a database table using JAVA annotation. The cache management finished the structure of the cache and how to manage cache servers by consistent hashing. The data access finished how the CRUD are implemented in databases and cache, and finished the implementation of the database read/write separating and how to solve the master-slave delay using distribute cache. As the core of framework, the processing capacity of the data access will affect the system's performance, so to enhance the performance of the database and cache operations become critical. Finally, the data reporting module is finished to help analysis the performance and cache usage of the framework by outputting many datas while running. The framework also improved the performance greatly through the JAVASSIST bytecode framework and multi-thread.All of the requirements of the framework was co mpleted currently, and the functional testing and performance testing was also done by simulating online request. It is now on-line and stable running for some time, and in the case of a large amount of concurrent requests the framework can also greatly im prove performance, and reduce a lot of the memory of the cache server.