The Design And Development Of Rendering Farm Based On Domestic Super Computer

With the rapid development of film and television industry, movie pictures are becoming more and more beautiful and realistic. However, many screen cannot be completed by normal shot, only be made by post-production, like modeling, special effects, rendering, etc. The increase of film resolution brings to the growth of single frame’s rendering time. Taking a common 4k video as an example, rendering a single frame will take hours to 10 more hours if we make use of mainstream workstations, while the more complicated scene that has numerous light sources and special materials will take tens of hours. And nowadays a great of films are made into 3D effect, which is needed to be divided into right and left eyes to render respectively. Although the computing power of stand-alone device is increasing rapidly, it has been unable to meet the demand of film industry. Therefore, cluster rendering is getting popular, which is often called render farm.Currently, the render platform built home and aboard is mostly based on ordinary small cluster and rarely thought about scheduling policy, and the resource partitioning and task allocation is completed by staff manually more often. In addition, it’s not optimized for rendering, a special application, and each node renders a single frame. While rendering is both high-density computing and data-intensive, whose memory usage is also high, and rendering one shot can have large amounts of shared data. Considering these characteristics of rendering, building the supercomputer-based rendering platform seems particularly appropriate, because the ability of connection between supercomputer’s nodes is much more powerful than the common cluster.The main work of this paper is building the high-efficient rendering platform by domestic supercomputer cluster. For the usage of cluster is complex, we developed a GridSphere frame-based portal website of rendering system, whose design refers to the general flow of online rendering, like uploading scene files, submitting rendering tasks, viewing and downloading the rendering results. Also it could hide the complexity of the cluster, manage the users and accounting.In this paper, aiming at the large amounts of shared data appeared in rendering one shot, we redesigned and implemented render units and data management software. The render unit provides a framework for running of data management software by direct dispatching render engine. While the data management software optimizes read and write of these shared data, reduces visits and the amount of visit data, avoids I/O congestion, and provides transparent access to the engine. Designing render unit and data management software is to make render engine run on this supercomputer platform more efficiently.The other work of this paper is to transplant the entire system to domestic supercomputers. The development of the system is completed in Shandong University Digital Media Center, however, there are quite differences in operating system, processor architecture and library files between supercomputers and ordinary cluster, therefore we have transplanted to The Light of Taihu in Wuxi Supercomputing Center and Tianhe-2 in Guangzhou Supercomputing Center, and have tested on the parallel scale of more than 300000 cores.The render engine mentioned in this paper is RenderWing, which is self-developed by Shandong University and conforms to the RenderMan Specification. What has done of this paper is part of the national 863 project,‘Development and application of realistic animation rendering system’.