Research On Virtual View Rendering Method Based On GPU And RGBD Data

With the continuous development of multimedia technology and people’s visual experience higher pursuit, traditional video of 2D can’t meet the public’s requirements of spiritual and cultural. Users can choose and operate the stereo multimedia at any angle through interaction, and then obtain the immersive experience. In recent years, along with the rapid development of relevant technology such as computer, network, image acquisition, free viewpoint video has become the growing trend of multimedia domain. Virtual viewpoint rendering technology, as the core component of free viewpoint video, has become the bottleneck of the development of free view video technology.According to whether to establish the three-dimensional geometric model, virtual view rendering technology can be divided into two types: rendering based on the model(MBR) and rendering based on image(IBR). The MBR method needs to build 3D geometric model, thus the complexity of rendering will increase with the complexity of the scene, and the large calculation of the method result in the higher requirement of the display device. The IBR method does not need to build 3D geometric model of the scene in the process of rendering, so the rendering speed has nothing to do with the complexity of the scene. This method can obtain directly virtual viewpoint image by interpolating or remapping. Virtual view rendering based on depth Image(the Depth Image-based Rendering, DIBR), as one kind of IBR, has become a research hotspot in the current virtual view rendering field. It can greatly reduce the network traffic and obtain arbitrary viewpoint images only by using the depth information, the internal and external parameters of the corresponding reference cameras. Unfortunately, the DIBR method will cause holes, artifacts, stacking, cracks and rendering efficiency, etc. problems. For example, the holes will appear when a wide range of information miss in the virtual viewpoint image; the artifacts will appear when foreground contour is rendered in the background; stacking will appear when background visible while foreground invisible; cracks will appear caused by the missing of local image information; and lower rendering efficiency obstruct the realization of real-time rendering, etc.In this paper, the problems above were studied delicately. On the crack problem, this paper proposed the adaptive median filtering algorithm. By modifying the pixels of cracks adaptively, it is effective on filling cracks, and meanwhile keeping the details of the image. On the artifact problem, this paper proposed a hierarchical mapping algorithm based on the depth image, through mapping the foreground and background respectively, and covering the background with foreground. The data set “Breakdancers” and “Ballet” of Microsoft Research Asia are used to show the results of classical rendering methods in this paper. The experimental results show that the algorithm proposed in this paper has obtained excellent results in terms of both subjective evaluation and objective evaluation. By concurrently processing 3D mapping and image fusion phases, a parallel rendering method based on GPU is introduced. Using CUDA C programming, the high density calculation processes on GPU, system has a great improvement compared with only on the CPU processing.In order to improve the efficiency and quality of the virtual viewpoint rendering, a prototype system of virtual viewpoint rendering with stronger practicability is established in this paper using C++ and CUDA C. An improved method based on GPU is introduced to solve the problems that seriously influence the virtual viewpoint image quality, which will promote the further development of the virtual viewpoint rending technology.