Remote Rich-Media Presentation

In the recent decades, the computing model has been moved from the centralized mainframe computing model to the more distributed desktop computing model. However, as the personal desktop computers become ubiquitous and the amount of the personal computers becomes larger and larger, the total cost used to own, maintain and upgrade the computers is becoming unmanageable. As a result of the rising-cost problem, there is an increasing movement to a more centralized computing model. The remote-access computing model is one embodiment of such a movement.Though the remote-access computing model is similar with the centralized computing model in the mainframe era, modern users are not satisfied using ASCII texts to represent the inputs and outputs of the applications any more. The remote-access computing model must support the rich-media environments to meet the users' expectation. The core technology to achieve a rich remote-access experience is the remote rich-media presentation. Remote rich-media presentation makes the rich media (e.g., the graphics user interfaces, multimedia display,3D graphics display, etc) at the remote server side real-time transmitted and presented to the clients, and meanwhile supports low-latency interaction with the end-users.With the rapid developments of the general computing processors (e.g., CPU) and the graphics processors (e.g., GPU), from the word processing application to high-frame-rate 3D graphics games and multimedia applications, the rich-media content is increasingly becoming diverse and its data amount becoming huge. The complicated rich-media content brings us many challenges to achieve efficient compression, transmission and interaction for rich-medias. In addition, in the environments of the heterogeneous networks such as the Internet, remote rich-media presentation has to couple with different bandwidths, different transmission delays and bandwidth fluctuation. Moreover, the client devices are usually different on the processing ability, the link bandwidth, the display resolution and etc. How to make the remote rich-media service accessible to these various devices is also challenging.This paper has conducted an in-depth investigation on remote rich-media presentation. We proposed a high-performance remote rich-media presentation system and proposed several innovative techniques to further improve the presentation experiences, reduce bandwidth/computing-resource consumption, improve the adaptation to the heterogeneous networks/devices, and improve the user interactive experiences. The main contribution of this paper covers several aspects, which is listed as follows.First, we proposed a high-performance remote rich-media presentation system. In this system, we developed a low-level compression-friendly architecture to represent the rich-media display information. Within the model, we further proposed an efficient and effective compression scheme for screens with compound contents. The proposed screen codec is based on fixed-blocks and employs adaptive compression mechanisms for different categories of contents. To provide users high-fidelity interactive experiences, we devised a smart option of fitting the active window to the visible region of the client. We also proposed an adaptive buffer control mechanism to adapt to the various network bandwidths. We evaluated the performances of the proposed system on real applications over a wide range of network environments. The experimental results show that the proposed system delivers superior performances compared with the previous remote rich-media presentation systems.Second, we proposed an efficient screen encoding solution for screen sharing to multiple clients. This solution supports one-pass encoding to generate multiple bitstreams with different bitrates. The one-pass encoding algorithm enables the host to only involve in one-pass encoding process for multiple bit-rates, and as a result the computation complexity of the host is decreased significantly. Specially, in the computing resource constrained environments, our algorithm can significantly improve the screen-sharing performance. Moreover, our encoding solution can also support the fast transcoding at the data center to support large-scale clients that access the screen-sharing service at the same time. Experimental results demonstrate that the proposed fast transcoding can greatly accelerate the processing.Third, we proposed an algorithm to employ the GPU to accelerate the screen-encoding process in the remote rich-media presentation system. In this algorithm, we leveraged all the computing modules except the entropy coder into the GPU in order to utilize the parallelism of GPU. Results indicate that the proposed GPU-assisted screen encoding can significantly accelerate the compression process at the remote server side.Forth, we proposed a fast video transcoding scheme to deal with the video content in the rich-media that is presented from the remote server to the client. We proposed a rate-distortion optimization model both for the pixel-domain and transform-domain fast transcoding. In this model, the distortion, defined as the difference of two predictions which come from two different sets of motion vectors but the same reference, is demonstrated approximately linear to the motion vector MSE. So the distortion can be estimated free from SAD/SSD, and the complexity of rate-distortion optimization is significantly reduced as well. We applied the transcoding scheme to H.264/AVC. Experimental results show that our proposed transcoding schemes in pixel/transform domain along with the proposed rate-distortion optimization method provide better performances in terms of trade-off between high performance and transcoding speed.Fifth, based on our proposed rich-media presentation system, we built a proxy-based rich mobile web browser. We used the server-side web parsing and rendering to leverage the browser computing logic. We used a composite screen format to represent the display of the web content, incorporating the web background screen and the dynamic web elements. And then we employed a slice-based screen encoding to efficiently compress the web background screen and update the screen to the client. We also used a progressive screen updating mechanism to improve the page latency. Besides the display screen of the web content, we also send the side information of the web elements to enable the designed element-level interaction mechanisms. The experimental results show that our browser can achieve the superior browsing speed, compared with the native browser and yield much better visual quality than the existing proxy-based browser.