Video Frame Synthesis Based On Deep Learning Method

If two consecutive frames in a sequence of video frames are given,video interpolation problem is predicting one frame between them by the information of given frames.The conventional video interpolation method relies on estimating motion information between two frames to synthesize intermediate frames.A classic example of this kind of motion information,called the optical flow,is defined as the motion vector distribution for each pixel in the image.In addition,as the research continues to deepen,people gradually find that it is not enough to obtain motion information.In order to restore the picture more finely,some post-processing operations are needed.In recent years,deep learning methods have developed rapidly,especially in the field of computer vision,and has achieved remarkable results.A natural idea is to introduce deep learning into the field of video interpolation.To solve this problem better,we need to combine human visual prior knowledge,some traditional visual theory and deep learning methods to design a coupling system.The system takes advantage of the strengths and advantages of each module.Our method mainly includes four steps: the first is to pre-process the given frames,the second is to improve the estimated flow distribution,the third is to generate two masks for synthesizing the intermediate frame,and the fourth is to post-process the synthesized image.The first step of the preprocessing operation can provide more information for the neural network,including some motion information between two frames.Each step of the last three steps is primarily a sub-network module.The second step mainly estimates flow distribution which is more helpful for interpolation.The third step relies on the neural network to judge the proportion of the front and back frames,and the fourth step can compensate some fine structures to obtain the final result.The experimental results on some public data sets prove that our method can achieve the best level in the field,and obtain numerically and visually excellent results.By analyzing the effectiveness of various modules,we can find the selection of the loss function,the training dataset and the basic network will have a great impact on the results,and the above three sub-network modules will certainly help the improvement of the results.The application of video frame interpolation technology with good results is very extensive,and it has received wide attention in the industries of film and television,animation or games.Improvements to the interpolated frame technology can help repair corrupted video,increase the realism of the video,and reduce the video bandwidth requirements.