Research On Video Interpolation Based On Deep Learning

The video frame rate conversion technique is a technique for reconstructing an intermediate frame by using correlation information between adjacent two frames in the video and applying interpolation.Since the technology can remove redundant information in encoding and reduce the frame rate during video transmission,and reduce the amount of data transmitted by the video network,it can be applied to video compression or enhance video continuity.The traditional video interpolation method mainly includes two steps,namely optical flow estimation and pixel synthesis.In this method,the effect of video interpolation technique often depends on the quality of optical flow estimation,and the process of optical flow estimation is susceptible to obvious errors caused by occlusion and blur.With the development of deep learning,the video interpolation technology based on deep learning has also made new breakthroughs.Some researchers have made some success in using video convolutional neural networks to try video interpolation.In this thesis,the optical flow estimation and the deep learning in the traditional method are combined,and an end-to-end convolutional neural network model is proposed,which combines motion estimation and occlusion processing.We first use the improved FCN network model to calculate the bidirectional optical flow between the input images,and obtain two warped images based on the estimated bidirectional optical flow information and the input image wrap operation.To solve the occlusion problem,we use another GridNet network.The model re-estimates the bidirectional optical flow information of the image and predicts the visibility of the pixels of the interpolated frame,and finally linearly fuses the estimated information with the original image to form an intermediate frame.In this thesis,we also tried a variety of loss functions,and finally determined the loss function that weighted various loss functions such as L1 loss,perceptual loss,wrap loss,and smoothness loss.The experimental results show that the video frame interpolation network structure proposed in this thesis can effectively improve the quality of optical flow estimation and improve the occlusion problem,and can generate intermediate frames with realistic,natural and better quality.