| With an object-relational mapping (ORM) tool, the software developer does not have direct control over the structured query language (SQL) calls to the database and thus relies on ORM decisions regarding how the database model is accessed. These database access strategies are critical to an application's performance since databases are typically remote from the application's runtime. In addition, an application's persistence data model may be complex, thus requiring more intricate SQL then simply retrieving data on a table-by-table basis. Overall, while an ORM relieves the developer from writing code, the ORM can also introduce performance issues if the ORM is not able to generate efficient SQL, manage transactions, and provide an adequate caching mechanism.; This research formulated an ORM performance evaluation framework by defining a set of performance test cases that are based upon common database access scenarios. The research then developed an implementation of the framework using a selected number of Java ORMs in order to evaluate the ORM processing time, database access calls, and the use of object caching within the ORM. The selected ORMs included two Java Persistence API (JPA) ORMs, Hibernate and OpenJPA, and two Java Data Objects (JDO) ORMs, JPOX and Speedo.; The performance test cases revealed significant performance differences between the selected ORMs. The JPA implementations were better performers when compared to the JDO implementations, with Hibernate being the overall best performer of all ORMs. The ability of an ORM to create more complex SQL provided significant performance gains when eager loading objects, bulk loading related objects, and performing batch inserts. The test framework also indicated that caching had a dramatic impact on reducing processing time when multiple calls are made to retrieve objects. In fact, caching was as important as efficient SQL generation when evaluating the processing times of the performance test cases.; The results indicate that ORMs exhibit considerable differences in terms of performance and database access, while all executing the same set of test cases. The application of a common set of benchmark performance test cases, as defined and implemented in this research, can be undertaken by the ORM community in order to provide an objective process for evaluating ORM performance. Such a benchmark would provide ORM users with insight into how the ORMs implemented common data access scenarios. This information would then allow developers to better select ORMs for their respective application contex. |
