Gigapixel Interactive Broadcasting System

With the rapid development of UHD,VR and AR,video viewing is undergoing an evolution from high definition(HD)display to ultra high definition(UHD)display.Gigapixel video breaks through the visual limit of human eyes in imaging resolution.It can capture the whole scene and the local details,so both large field of view and local details can be ensured.But with the limitation of network bandwidth and client decoding ability,the gigapixel broadcast is facing great challenges.This paper focuses on how to realize gigapixel video broadcasting.The main contents and innovations include:1.Collect and produce gigapixel panoramic video.Aiming at the natural contradiction between the imaging resolution and signal-to-noise ratio of a single camera image sensor,and the problem that the existing billion-level pixel sensor cannot adapt to the high framerate.This paper proposes a method of acquisition and splicing based on parallel cameras to improve the resolution and ensure the image quality.In order to ensure the efficient real-time mosaic of multi-camera images,the system uses parallel computing to set the parameters of video stream,such as resolution,bitrate,white balance and exposure,and generates panorama by mapping the position between local high-definition images and a low-definition wide-angle images on the client side.So,the whole delay of the system is reduced and the system can achieve real-time gigapixel broadcasting.2.Transmission and forwarding of video streams.Real-time transmission of gigapixel video data requires the support of high bandwidth network,which brings serious challenges to video transmission.For this reason,the system compresses and encodes the massive video data before transmitting it,and builds a central control platform to preview and focus the real-time streams of all cameras.This platform collects the video data of all cameras,thus realizing the unified push and forward of video streams.3.Multiscale hierarchical block video processing.In order to further reduce the demand for network bandwidth for gigapixel video transmission,based on the characteristics of human vision,the system adopts the high-quality transmission of visual field area content and low-quality transmission outside the field of view,thus can achieve both large field of view and local high-definition detail video.Specifically,the 19 4K videos collected by system are sampled at 3 resolution levels.For each 4K resolution original video,the system re-encodes 16@960x540 slices by H264.When the client loads,it only needs to transmit the required resolution level video slices.So the client just needs 20Mbps to watch the gigapixel video,reducing the network bandwidth requirements.4.The interactive display of gigapixel video.In order to meet the video viewing needs of different clients,the system decodes multiscale video stream and supports video switching and zooming.To ensure the consistency of scene when the video is dragged,this paper caches the data outside of the view in advance.For now,users can open the client to watch live video at anytime and anywhere both in PC and mobile.The overall delay of the system from the acquisition terminal to the client is less than 500 ms,which meets the demand of live broadcasting.The interactive gigapixel broadcasting proposed by this paper would be used in sports,meeting and so on in the future,enriching the form of live broadcasting and leads the UHD broadcasting in the era of big video.