Hybrid mesh/image-based rendering techniques for computer graphics applications

Although visibility (or occlusion) culling techniques have been extensively studied and adopted for fast rendering of occluded objects in a scene, rendering of botanical models poses a major challenge that is not well addressed before. That is, objects in natural scenes tend to be large in the number of counts (e.g. trees), yet they are usually made out of small primitives (e.g. leaves) which make traditional occlusion culling a non-trivial task. In this research, we present two techniques to simplify tree leaf objects for their fast rendering while retaining their visual resemblance to the original model.;The first technique, called the camera-imaging-based approach, is to prioritize leaf objects based on their geometrical relationship to multiple camera viewing angles. First, we consider a single camera viewpoint. Under a budget constraint of how many leaves shall be rendered, we select leaves with higher priorities for rendering. We partition camera's projected screen space into several small regions to ensure that each region is guaranteed to be filled with a certain number of leaves. Each region selects higher priority leaves by utilizing leaf objects' spatial relationship to the camera. Then, an iterative algorithm is proposed to allocate leaf objects at different viewing angles. The algorithm chooses different viewing positions for leaf allocation at each iteration with the goal of maximizing the resemblance to the original tree model at all possible angles. Three criteria taken in consideration for viewing angle selection are: the distance, leaf normal, and unfilled pixel.;In the second technique which is called the human-vision-based approach, we develop a human-centric tree simplification technique by exploiting visual saliency of the human visual system (HVS). Saliency maps such as the intensity, orientation, and color of an image are critical to human perception and can be used to serve as the design guideline of the simplification algorithm. Built upon the first technique, a subset of polygons is chosen for rendering to speed up the rendering process. Rather than mapping each polygon with the same leaf texture, we scale these polygons and map a different number of leaf images to them based on polygon's density and position on the tree.;Four objective performance metrics are considered for performance evaluation; namely, pixelwise comparison, block-level density comparison, the Gabor-filtering-based metric and the saliency-map based metric. In the pixelwise comparison, the pixel-wise difference between rendered images of the original and simplified tree models is computed. In the block-level density comparison, the different between pixel densities at a block of a certain size is calculated to ensure that errors reflect distortions at a local image level. The third one is to use the Gabor filter to analyze the orientation and intra pixel relationship. Finally, to characterize human visual characteristics, saliency maps of rendered images of simplified and original models are generated and a mean-squared-error (MSE) is calculated to quantify the resemblance between them. Thus, it is called the saliency-based metric. We also conducted subjective evaluation tests by asking human subjects to rate the resemblance of simplified tree models to the original ones. It is shown that subjective evaluation and the saliency-based objective performance metric reach consistent results. Experimental data are also provided to demonstrate time efficiency of the proposed techniques at the cost of little visual degradation.;After considering different techniques in simplifying a tree model, we take a step further to enhance the rendering performance. That is, we examine a hybrid mesh/billboard approach which renders an object either as 2D image or its original 3D format. First, we investigate the tradeoff in rendering objects with either the mesh or the billboard representation. While billboards provide faster rendering performance in a scene with multiple objects, it incurs some processing overhead that makes such a representation not the best choice in some cases. A design guideline on the optimal balance of using meshes and billboards is studied. Three major parameters used in the tradeoff analysis are: the distance between the camera and the object, the billboard update time, and the scene complexity. We examine the relationship among the three parameters and determine the best tradeoff based on system requirements. Finally, other parameters that may contribute to changes in the rendering resource requirement are also briefly described.