Optical Flow Estimation Using Dilated Convolution Deep Network

Optical flow is the projection of the three-dimensional motion on imaging plane,and is the apparent motion of the luminance pattern within the videos.The optical flow field contains not only the motion information of objects and backgrounds in the image,but also rich threedimensional structural and surface information.Therefore,optical flow estimation is a significant problem in computer vision and pattern recognition,and has many applications,e.g.,videos segmentation,action recognition,motion detection and tracking,meteorological analysis,autonomous driving.Inspired by the successes of deep learning,optical flow estimation based on deep learning has a good performance in terms of accuracy and robustness.However,when there are complex situations such as large-displacement motion,motion blur or occlusion,and non-rigid deformation,the performance of the existing optical flow estimation algorithms is significantly reduced.In this paper,we propose two improved models to learn to predict the optical flow field from videos under large displacement motion,and these models are based on the existing CNN algorithm for optical flow.1)FlowNet-based optical flow optimization model.Aiming to improve poor performance of optical flow estimation algorithm based on convolutional neural network,we propose a twolevel optical flow optimization model by using multi-scale residual learning.The first-stage network mainly predicts the results of the motion features in a pair of images,and the refined network in the second-stage uses the residual information to improve the optical flow results generated by the first-stage network.2)Optical flow estimation using dilated convolution.An optical flow estimation model based on the dilated convolution depth network is proposed to improve the situation that the edge of the optical flow is not smooth and the average endpoint error(AEE)is larger.Multiple parallel dilated convolutional layers are adopted in the estimation model to simultaneously extract the relevant spatial feature representations of the input images,which improve the situation that the uneven edge of optical flow prediction result,and make better the accuracy of optical flow estimation.Based on the three public benchmarks datasets including Middlebury,KITTI and MPI Sintel,compared with the popular FlowNet,SPyNet and FlowNet2,the two optical flow estimation models above are simulated The experimental results show that the optical flow estimation model based on the dilated convolution depth network proposed in paper can accurately predict the optical flow and improve the accuracy and robustness of the estimation of optical flow.