The Deep Learning Approach To Optical Flow Evaluation Based On Spatiotemporal Information

Moving object detection is also known as tracking problem,which is an important part of computer vision.There are many related methods,among which optical flow method describes the change trend of gray value of pixels in the image,that is,the velocity vector field of pixels.In the past few years,deep learning has achieved good results in computer vision,natural language processing and other issues.As the theory of convolutional neural network(CNN)matures,the related research gradually goes into the field of optical flow characteristic map evaluation.In this paper,a network substructure called "Fast Optical Flow Unit"(FUOF)is proposed,which enables the original network to evaluate the optical flow and generate the feature map by a fast and robust method.The FUOF method proposed in this paper takes FlowNetC as the improvement object,and the original FlowNetC has a pioneering use of CNN to carry out end-to-end learning of optical flow assessment tasks.In this paper,FUOF,which is based on the properties of optical flow itself,can guide the network to evaluate optical flow and enhance the adaptability of the network to different motion trends.In order to save the computing time,we propose the Extend Sobel operator to reduce the time of secondary fusion after extracting spatial information using Sobel operator.At the same time,in order to make full use of space and time information,this paper discusses two methods of space-time information fusion,FUOF-Sum and FUOF-Concat.The idea of this paper is confirmed by the experimental results of MPI Sintel and Flying Chairs datasets.According to the research results of this paper,FlowNetC with FUOF has a strong competitive advantage in accuracy,running speed and robustness.In addition,this paper also compares the results with the advanced lightweight optical flow evaluation network LiteFlowNet.In terms of accuracy,the improved FlowNetC effect of FUOF proposed in this paper is more accurate.