Research On Video Frame Interpolation Metho

With the development of high frame rate video website,the demand of video frame interpolation is increasing.Video frame interpolation is designed to synthesize virtual frames from two real input frames in order to improve video frame rate.Most researchers interpolate videos by using Optical Flow and Convolutional Neural Network.CNN usually fuses two input frame by using a small kernel to interpolate videos.The larger the kernel is,the better the details of the input frame can be recognized,and the more computing cost the method generates,which is why most researchers used small convolution kernel before.We often estimate bi-directional Optical Flow and warp the input frame to synthesize the intermediate frame and avoid the void hole of the intermediate frame.But the OF hardly deals with the occlusions,and the frame interpolation results are not accurate.In order to make up the flaws of CNN and OF frame interpolation methods,we proposed three different solutions in this paper:(1)A video frame interpolation method based on separable convolution with super kernel.Due to the fact that the larger the kernel is,the more accurately details and texture of the input frame can be recognized,this model uses CNN with super kernel to obtain more details.We design a whole new depthwise separable convolution which can make different convolution layers running at the same time in order to balance the longer running time caused by the increasing of the convolution kernel size by integrating the advantages of separable convolution.Compared with the traditional CNN interpolation methods,this model can recognize image texture and motion more accurately,and also keeps the computing cost within an acceptable range,so as to effectively improve the accuracy of video frame interpolation result.(2)A video frame interpolation method based on scene flow.This method interpolates videos with OF.However,most traditional OF interpolation methods never consider the movement of the objects and cameras in the scenes,those static objects are also computed by the interpolation methods,which is a waste.We use OF to distinguish static rigid and dynamic non-rigid scene flow,and carry out different processing operation for them.We use external calibration to evaluate the camera trajectory and pose,and combine the scene flow to synthesize the intermediate frame.Compared with the traditional OF interpolation method,this model can greatly reduce the computation cost.Due to the fact that we consider the motion and pose of the camera,the model can get more accurate interpolation model.(3)A video frame interpolation method for processing videos which contain high light source.Previous OF interpolation methods usually produce serious blur while interpolating due to the fact that high light source effect the OF method’s judgment of the object occlusions.Therefore,we design a new frame interpolation method for video with high light source.Firstly,we balance the light intensity in different areas of the input frame.Secondly,we provide the processed image and its light intensity feature to the frame synthesis network with depth-aware model,so that we can get the intermediate frame and its light intensity features.Finally,we restore the light intensity of the intermediate frame with its feature.Compared with the traditional frame interpolation methods,this model adds image intensity processing module,thus reducing the errors and blurring while processing such videos with high light source.