Research On Real-time Optical Flow Algorithm Based On FPGA

Optical flow algorithm is one of the key research topics in the field of artificial intelligence such as computer vision,image processing and machine learning.With the aid of optical flow method,the motion parameters of the target in the video sequence can be obtained effectively,and the tracking analysis and behavior understanding of the moving target can be realized.Optical flow method has been widely used in a variety of image and video related fields.In terms of theoretical research,the optical flow estimation theory of image sequences has been proposed as early as the 1980 s,and the typical representative is the Lucas-Kanade(L&K)optical flow algorithm based on differential method.The algorithm uses the grayscale image sequence or its filtering form of time and space gradient function to calculate the pixel velocity vector.However,the high computational complexity of these algorithms limits their application and development in the field of industrial motion detection,meteorological warning,UAV detection and other fields.Compared with PC,DSP and other hardware platforms,FPGA has advantages of high portability,fast running speed and low power consumption.In this thesis,high-performance FPGA is used to complete the hardware implementation of the algorithm.Based on the above realization ideas,this thesis proposes an improved real-time flow algorithm based on FPGA.Based on the L&K optical flow algorithm,the algorithm structure is optimized to improve the algorithm accuracy,and the algorithm hardware module is improved to reduce the resource consumption in FPGA chip,so as to improve the real-time performance of the algorithm.The main work involved in this thesis is as follows:(1)An improved real time flow algorithm based on FPGA is proposed.The main process and module design of the algorithm include: 1)Preprocessing the grayscale image,selecting the median filter to eliminate a lot of salt and pepper noise and other noise points with a large grayscale gradient;2)An external DDR memory control unit is designed to store the image sequence data obtained by denoising;3)An optical flow calculation unit is designed to calculate the estimated value of optical flow image sequence of each layer;4)In order to facilitate display and visualization analysis,a VGA display module is designed to observe the image results processed by this optimization algorithm.The internal units of the above optical flow computing architecture are all designed and implemented based on FPGA platform.(2)Optimal design of optical flow accuracy.Since the application scope of L&K optical flow algorithm is limited to the small displacement movement of objects,the assumption of constant optical flow can only be satisfied in the minimal neighborhood range of image pixels.If there are high-speed moving objects in the image,this assumption will not be valid,so it is difficult to meet the application of various scenes.To solve this problem,this thesis designed a three-layer ladder optical flow calculation framework to improve the accuracy of optical flow algorithm,which effectively improved the problem of insufficient computational accuracy of classical L&K algorithm in large displacement motion scenes.(3)Optimal design of optical flow aging.In the process of gradient operation,considering the need for real-time processing in FPGA,this thesis improved the gradient operation process of the algorithm,thus effectively improving the operation efficiency of the hardware platform;In addition,when the product operator is convolved,the convolution results under different variances are compared and analyzed,and the optimal variance value is selected for optical flow calculation.According to the above process design and optimization strategy,experimental verification was carried out on Middlebury optical flow test set and real video sequence respectively.Experimental results show that in terms of algorithm optimization,the average Angle error(AE)of the optimized optical flow algorithm proposed in this thesis is reduced by 2-3 times compared with the traditional L&K algorithm in Grove and other 5 classical data sets tests.Hardware implementation: The real-time time flow algorithm in this thesis has higher operating accuracy on FPGA platform than on PC or DSP platform,and lower overall hardware resource consumption.It can realize real-time video target tracking with640×480 resolution of 100 frames /s,and obtain more real-time and accurate optical flow results.