Design and evaluation of a multimedia computing architecture based on a three-dimensional graphics pipeline

Over the last decade, dynamically-evolving multimedia applications have required ever-increasing computing power. Thus, several types of domain-specific processors for multimedia, e.g., mediaprocessors and graphics processors, have been introduced. Mediaprocessors are typically used for audio, image and video applications whereas graphics processors are targeted for 3D graphics applications. With the innovation and integration of media objects in advanced multimedia applications, the application domains of mediaprocessors and graphics processors are merging, thus the importance of unified architectural support for both graphics and non-graphics multimedia computing has increased. While several approaches based on vector or VLIW architectures have been pursued, they are not as effective as dedicated graphics pipelines for high-performance 3D graphics. We have explored a new programmable computing architecture based on a 3D graphics pipeline so that different types of media objects are efficiently and seamlessly handled in a programmable fashion. The proposed architecture is advantageous over vector or VLIW architectures in that it can provide high 3D graphics performance since it is based on a graphics pipeline. In addition, there is a large amount of computing and memory resources, which could be effectively reused for non-graphics multimedia computing with proper extensions. Specifically, we extend the texture filtering unit in the base graphics pipeline to a programmable processing unit so that it can execute instructions suitable for both graphics and non-graphics multimedia computing. Several other units, e.g., rasterization and texture mapping units, are also extended to a data streaming unit to efficiently deliver a large amount of multimedia data. With cycle-accurate simulation of benchmark functions, we have verified that the proposed architecture outperforms MAP-CA, a modern powerful mediaprocessor currently available, in imaging and video processing by a factor of 1.2 to 15.9 in most cases. In addition, the original 3D graphics performance is not compromised even with the additional pipeline stages for the extensions because the additional pipeline stages result in longer pipeline latency but similar throughput with our proposed latency hiding technique that dynamically utilizes multiple configuration registers.