Research And Implementation Of Key Technology For Remote Rendering Based On Video Streaming

The continuous development of rendering technology makes it possible to visualize large and complex data,and its high computing resources and strong data confidentiality make it difficult for people to easily use complex rendering systems through everyday equipment.Remote rendering is an effective solution to this problem.One of the important factors restricting the development of remote rendering technology is the delay of network communication and the poor real-time interaction due to network bandwidth limitations.Remote rendering using video stream compression technology can achieve better interaction with less bandwidth compared to remote rendering schemes that transfer images directly.This paper proposes and designs a WebRTC-based video streaming remote rendering system,expounding the key technologies of each part.Based on this,two general optimization strategies are designed and implemented with the transmission bandwidth and client display frame rate as evaluation indicators.The specific research content of this paper is as follows:(1)Through the analysis and research of remote rendering related technologies,a video streaming-based remote rendering system based on FFmpeg and WebRTC technologies is implemented,which includes key modules such as rendering result extraction,data caching,video encoding,communication establishment,and congestion control.(2)In order to reduce the transmission bandwidth of the video stream,a bandwidth optimization method based on the region of interest is designed.Different from the nature that information between picture frames are isolated,the encoded video frames compress the redundant information carried by the frames.On the basis of not changing image quality of the user's attention area,the bitrate of video stream can be lessened by reducing the information carried by the non-attention area,thereby achieving the purpose of optimizing the bandwidth.(3)In order to improve the framerate of client video playback,a Lucas-Kanade algorithm-based video fluency optimization method was designed.The framerate of the rendering system itself may not be sufficient to meet the requirement of smooth display.Before the rendering result is compressed into a video stream,the framerate can be increased without changing the original rendering system through image interpolation.Lucas-Kanade algorithm is an algorithm for fast computation of optical flow.Based on this algorithm,interpolated frames can be achieved between images to achieve the purpose of improving client display fluency.