Real-time Performance Of Optical Flow Computation

Optical flow filed is defined as motion information from the image sequences. It canprovide motion information and three-dimensional structure information of visual movingobjects. Optical flow computation is used to do the estimation of motion by extracting eitherinstantaneous image velocities or discrete image displacements. It is widely used in manyvideo/image based applications such as motion estimation, video compression and objectdetection and tracking. Thus Optical Flow is one of the central problems in patternrecognition, computer vision and image processing.However, high-quality optical flow algorithms are computationally intensive. Slowcomputation limits the applicability of optical flow computation in real-world applications,especially in embedded systems. For instance, on a2.9GHz Intel i73520M Processor, theclassical method of Horn-Schunck (i.e., HS) needs2.3s to compute optical flow between twoframes when the image size is640*480. Moreover, the complicated methods of optical flowcomputation are slower so that it takes a few seconds or longer to processing. In particular,they are considered to be too slow for those tasks where real-time performance is needed.Thus real-time performance of optical flow computation with high energy-efficient is thefocus problem in this paper.Variational optical flow methods belong to the best performing and best understoodtechniques for estimating optical flow field. Firstly, three theoretical models of variationaloptical flow methods are studied. Secondly, the computational properties and performance ofdifferent variational methods are estimated on CPU, then the general work-flow of variationaloptical flow computation are refined. Thirdly, the high-level synthesis (i.e., HLS) languagetogether with traditional hardware description language are used to describe variationaloptical flow accelerator based on HS and Combine-Brightness-Gradient (i.e., CBG) models.Finally, the optical flow accelerators are working in the software-hardware Co-processingmode and implemented on diferent heterogeneous platforms such as the ZYNQ-7000SoCand the PC-FPGA platform with a Kintex-7FPGA respectively. Portability of the system isdesigned carefully for deploying it on different platforms conveniently.In this thesis, we provide a systematic toolkit for real-time performance of variationaloptical flow computation, present a FPGA-based optical flow computation system with highportability and energy-efficient in the software-hardware Co-processing mode. Takenresolution640x480as instance, the result shows that FPGA-accelerated CBG outperforms33x than the pure software vision on ZYNQ. The execution time is decreased from26.68s to0.82s, and the energy is only0.35J per frame. While12fps and0.38J per frame are achievedby this optical flow computing system when640*480image is used and optical flow for allpixels are computed on PC-FPGA platform.