Research On Highly Accurate Estimation Of Optical Flow

Optical flow can reflect the motion information of objects by extracting precise pixel-wise correspondences.As the fine-grained characteristic of its result,optical flow is commonly used in the area of action recognition,image registration and fusion,detection of population aggregation and auto driver assistance.Optical flow is the fundamental component of motion segmentation,analysis and recognition.However,optical flow in real natural environments is confronted with many challenges: the flow accuracy in the case of sophisticated texture and non-rigid moving objects is still not satisfactory;the large displacement caused by fast movement is still an open problem;the estimation of the correspondences in large areas of occlusion is ill-conditioning problem remaining to be solved;how to boost the algorithm efficiency in the case of high resolution frames is a key problem should be considered in actual engineering.To handle these problems,this dissertation deeply analyzes the techniques for highly accurate large displacement optical flow.With thorough validation on public thirdparty datasets,the proposed methods are among state-of-the-art optical flow algorithms both in efficiency and accuracy.Furthermore,the proposed methods are in the process of being deployed in a mobile phone project which is about high definition image fusion based on dual cameras.The main contributions of this dissertation can be summarized as follows:(1)existing filter-based methods can hardly survive in the case of large label space,and the accuracy is just not satisfactory.To handle these problems,we propose a highly accurate optical flow method based on superpixel tree.Built on a tree representation of image,we propose a two-step filtering framework by combining the advantage of local and non-local methods.The first step of the framework is a non-local filtering method based on superpixels.The resulting superpixel flow serves as the guidance of the second local filtering procedure.The proposed method can efficiently deal with huge label space which is common in optical flow estimation,resulting in a speedup of nearly 13 times on typical Middlebury dataset.The efficiency benefit is mainly contributed by the two-step scheme,which divides the entire label space and makes the method less sensitive to the size of the label space.Furthermore,superpixels in the framework can be processed independently,which makes the framework suitable for parallel platforms.(2)methods based on energy minimization are prevalent in the field of optical flow.How-ever,constricted by the continuous optimization techniques and improper assumption,these methods often fail to meet the case of large displacement.Although many scholars try to fix this problem using techniques of Nearest Neighbor Fields(NNF),the severe noise problem is the foremost obstacle due to the lack of regularization restriction.For this problem,we propose an efficient coarse-to-fine semi-dense matching method for large displacement optical flow.Combining random search strategy and the coarse-to-fine scheme,we can effectively deal with the problem of matching noise and lack of regularization,which are the main problems in traditional NNF methods.The method utilizes the advantage of discrimination on higher levels,and avoids the ambiguity problems on lower levels.Furthermore,the matching density is over tens of times the density of traditional SIFT,SURF etc.matching methods.(3)with the development of sparse matching and semi-dense matching,optical flow by direct interpolation from matching shows significant advantages in the case of large displacement and occlusions.However,existing interpolation methods are too sensitive to the input matching noise.Moreover,the matching noise is inevitable.To cope with this problem,we propose a robust interpolation method for large displacement optical flow.First,the interest scene is over-segmented to superpixels whose motion pattern are assumed to be affine motion models.Then,RANSAC techniques are applied to estimate the affine model robustly for each superpixel.This method can still steadily work under the noise level of nearly 40%.