Optical Flow Estimation Based Video Frame Interpolation In Complex Motion Scenes

Video interpolation is an important research direction in the field of computer vision,which is often used to improve the smoothness of video and create slow-motion effects.The video interpolation algorithm has a certain range of application,and it can easily introduce negative effects when interpolating frames for complex motion images,resulting in degradation of watching experience.Therefore,this research proposes a video interpolation algorithm with higher robustness in complex motion scenes,based on three improvement dimensions: more reasonable process design,more accurate low-level operators and special content processing.First,this study proposes a forward warping based single-module video interpolation algorithm based on the analysis and summary of existing procedure.The algorithm uses RAFT,a high-performance optical flow estimation network,as the motion estimation module,and achieves state-of-the-art quantization metrics after end-to-end training,and has better visual effects than previous frame interpolation algorithms.Second,this study further explores the forward warping operator used in the interpolation process,and proposes an warping based implicit depth estimation strategy based on reasonable empirical assumptions,and uses its estimation results as a priority mask to deal with the many-to-one conflict in the traditional forward warping process.This strategy can improve the visual effect of the frame interpolation algorithm in the scene of relative motion of foreground and background,and at the same time can bring about a gain of 0.2 d B in PSNR metrics.Thirdly,this study proposes to enhance the post-production content areas such as subtitles and logos in video to improve the visual perception by solving the jittering problem of the frame interpolation algorithm in these areas.The enhancement scheme is applicable to many common types of text and signs,with less parameters and computation increasement,and less coupling with the video interpolation algorithm.The quantitative index of this algorithm is close to state-of-the-art algorithms on the commonly used video interpolation test sets Vimeo90 K and UCF101,wh ile it maintains better visual effects in complex motion scenes,and has better ge neralization performance than existing algorithms.