Unilateral View Synthesis Method And Its Applications In Multiview Coding

2D video projects real-world's three-dimensional space onto two-dimensional space, effectively saves 2D scene information while losing the spatial information. To make up 2D video's realistic and spaciousness, 3D video technology came into being. Audience needs, commercial importance, technical support, powerful combination of the three makes 3D video technology rapidly develope in the short time since its emergence. Such as the "Avatar" and other similar 3D video works are sought after greatly. Therefore, the multi-view, stereoscopic video codec technology has become an international research focus on the areas of multimedia technology and provides strong technical support for a range of commercial products and purposes. Network bandwidth is gradually increased as the updating of hardware device.However, the increase of video data is apparently much faster compared to the increase of bandwidth, which highlights the increasing importance of data compression technology. Video compression technology continues to improve coding efficiency, makes the applications needing processing a large amount of data possible. Multi-view video is catched on the same scene by multiple cameras simultaneously from multiple angles. The amount of data increases with the number of cameras increases, which is far more bigger than a single point of 2D video data. In order to code multi-view video data, based on the traditional two-dimensional video coding technology, the mainstream method uses the correlation between views to further reduce the resuals and enhance coding efficiency. HHI proposed a layered B-frame prediction structure, and multi-view Video Coding Experts Group has listed it as international standards due to its easy structure and outstanding coding performance. Its basic idea is combining temporal estimation with inter-view estimation and using inter-view correlation to reduce prediction residual energy, which increases the coding performance then.In many applications, multiview data are usually indicated by a kind of 2D texture + depth multiview video presentation format. Depth information is the data describing the distance between the target and the camera lens. The structure can be well applied in the applications needing to group texture and depth for transporting and encoding. Based on texture images and depth maps, using warping transformation, one can synthesize one virtual view from the texture and depth of other views. There is much similarity between the virtual view and the view cathched by the camera and we can replace the original view with the virtual view in some applications, which is the main purpose of view synthesis. On the other hand, the virtual view can be used as an additional reference view to enhance the coding efficiency of current view, which is the ultimate principle of view synthesis prediction.Currently, the widely used view synthesis technique commonly takes the two views at both sides of current view to synthesize two virtual views and merge the two virtual view to remove the inevitable holes due to unilateral view synthesis. This paper presents a unilateral view synthesis technique based on interpolation and background modeling, trying to solve the problem how to synthesis the virtual view with only one referrence view. Subsequently, we take unilateral view synthesis technology into multiview coding to enhance the coding efficiency of mutlview even view data. Experimental results show that the method can be applied into the multiview video coding standards without obviously time-consuming.