Research On Frame Rate Up Conversion On3D Video

Nowadays,3D video has attacted a lot of attention due to bringing people morerealistic visual experience. So more and more3D videos begin to enter the market inthe form of3D film,3DTV program and3D cartoon. In order to better represent3Dvideo, a new3D video format which includes multi-view video plus correspondingdepth maps (MVD) was proposed. In MVD, the texture video and the depth map arethe descriptions of the same video scene. The texture video is the original2D video,yet the depth map provides the distance information between the objects and thevideo camera, so it’s a depth description of the scene. For further improving thevisual experience of3D video, frame rate up conversion (FRUC) can be performedon the texture video in3D video, which refers to inserting some new frames amongthe original frames to upconvert the frame rate of the texture video and make thevideo more fluent. The depth map in3D video which providing the auxiliaryinformation can help FRUC on texture video to improve the quality of theinterpolated frames. How to utilize the depth map to enhance the quality of FRUCon texture video in3D video becomes our study point. This thesis combinesconventional FRUC techniques on2D video and the characteristics of3D video, usethe correlation between texture video and depth map to investigate FRUC methodson texture video in3D video. We focus on motion estimation (ME), motion vectorfiltering (MVF) and motion-compensated interpolation and propose thecorresponding improved methods. The main work and innovations lies in:1. Proposed a depth-constrained bi-directional ME methodThis thesis analysed the ME process in Motion-compensated frame rate upconversion (MC-FRUC). Based on bi-directional ME, with the help of the distanceinformation in depth map, we proposed a depth-constrained bi-directional MEmethod and integrated it into FRUC on texture video.2. Proposed a depth-guided MV filtering methodThis thesis analysed the necessity of the motion vector filtering. Based on thevector median filter, with the guide of depth information and the analysis of thecorrelation between the texture video and depth map, we proposed a depth-guidedmotion vector filtering method to enhance the local consistence and depthconsistence of the motion vector field.3. Proposed a occlusion-considered interpolation methodThis thesis pointed out the occlusion problem in FRUC process at first andanalysed the drawback of the interpolation method under the occlusion problem.Then we proposed a occlusion-considered interpolation method. The proposed method utilize the motion correlation between texture video and depth map andchoose the forward reference block or the backward reference for interpolationdepending on the judgement of the occlusion with the help of the depth information.