Research On Methods Of Video Frame Rate Up-Conversion And Depth Map Inpainting For 3D Video

In recent years,emerging digital multimedia technologies have brought people a variety of cinematic experiences.Three-dimensional video(3DV)can provide spectators with immersion experience,received extensive attention and research.One of the main factors that influences the effect of 3D video is the number of viewpoints.As the number of viewpoints increases,the viewing stereoscopicity enhances and the viewing area increase,the amount of data also increases.In addition,in order to improve the display effect,3DV generally has very high resolution and high frame rate(HFR).Therefore the amount of 3D data is several times than traditional two-dimensional video(2DV).The huge 3D data and limited bandwidth impede the development of 3DV.In order to solve the problems of bandwidth limitation on 3DV transmission,this thesis extends the 2D video frame rate up-conversion(FRUC)method to 3DV,which uses the color image and the depth map to increase the frame rate in 3DV,so that the system can reduce the amount of transmission data at the sender and synthesize smooth HFR 3DV at the receiver.Since the 3DV FRUC algorithm requires high quality depth map,and the depth map acquired by conventional depth camera has a lot of noise and holes,thus the depth map inpainting method is also studied in this thesis.Besides,the original 3DV FRUC aims to reduce the transmission data,however,it is easy to be used to tamper the video format to achieve the purpose of increasing the click-through rate of the uploaded videos.In response to this problem,this thesis also proposes a video format tamper detection method.The main contents of this thesis can be summarized as the following:1.A method of 3DV FRUC based on depth map plus color image is proposed.This method utilizes the edge consistency between depth map and color image.Firstly,the block is classified into the edge block and the flat block in the motion estimation.Different motion vectors(MVs)are estimated according to the block properties to obtain accurate MVs.The MVs can be detected and corrected by a MV post-processing.With precise MVs,pixel points are compensated according to whether the pixel point is an edge point.In addition,the texture enhancement-based MV estimation criterion is applied therein,which facilitates the fast search of matching blocks.The algorithm can effectively improve the quality of the interpolated depth maps and color images.2.A depth map inpainting method based on edge constraints is proposed The method mainly includes two parts.The first part is the classification of depth holes,where the depth hole is classified and marked by the edge consistency of the colorimage and the depth map.The second part is the inpainting of depth map.Specifically,hollow pixels are filled by the bilateral filter based interpolation method,and then thedepth map is denoised by reliability based joint trilateral filter.This method can effectively improve the quality of the depth map taken by the Kinect camera.3.A video pseudo-frame detection method based on spectrum analysis is proposed.Firstly,the Local Binary Patterns based key frame detection method is used to segment the shot of the original video.Then a Discrete Cosine Transform(DCT)is performed on each video sequence,and difference between the original frame and the artificial frame is highlighted by filtering out the low frequency part of the image.The Inverse Discrete Cosine Transform(IDCT)is performed on the filtered image,and the edge intensity of each frame is calculated to obtain the edge strength of the frame sequence.Finally,the curve is binarized by the Huffman mean line,which locates the inserted fake frame and estimate the original frame rate of the video.The method can effectively detect the fake frames in videos which tamper the video format by FRUC.Experimental results shown that subjective visual effects and objective PSNR of our 3DV FRUC algorithm are better than classic algorithms.And the depth map inpainting algorithm can reduce depth map holes and noise,which improve depth map quality.The average detection accuracy of our fake frame detection algorithm is around 97.2%on the test video dataset.