A Study On Motion Estimation In Computer Vision

Motion Estimation is one of the fundamental problems in computer vision.It has broad applications in the fields of robot navigation,mixed and augmented reality,visual tracking,image and video processing,intelligent transportation systems and so on.Up until now,motion estimation is far from a solved problem,and it is still one of the active research topics in and beyond the computer vision community.This dissertation is dedicated to both camera motion estimation – including motion estimation for 3D and 2D cameras – and dense image motion for color images.We push the limits of the state of the art in various aspects such as optimality,robustness and flexibility.The main contributions are summarized as follows.A globally optimal 3D point cloud registration algorithm is proposed and applied to motion estimation of 3D imaging devices.The Iterative Closest Point(ICP)algorithm is one of the most widely used algorithm,however it is susceptible to local minima.Based on Branch-and-Bound(BnB)optimization,we present the first globally optimal solution to the registration problem defined in ICP.By exploiting the special structure of geometry,novel bounds for the registration error function are derived.Other techniques such as the nested BnB and the integration with ICP are also developed to achieve efficient registration.Experiments demonstrate that the proposed method is able to guarantee the optimality,and can be well applied in estimating the global or relative motion of 3D imaging devices such as 3D scanners or depth sensors.A globally optimal inlier-set maximization algorithm for color camera motion estimation is proposed.RANSAC is a popular algorithm to handle feature mismatches or outliers,with its goal being inlier-set maximization.However,RANSAC cannot guarantee the optimality,or even can never achieve the optimal solution due to the use of algebraic motion solvers.In this dissertation,we propose using BnB to seek for the optimal motion which gives rise to the maximal inlier set under a geometric error.An explicit,geometrically meaningful relative pose parameterization – a 5D direct product space of a solid 2D disk and a solid 3D ball – is proposed,and efficient,closed-form bounding functions of inlier set cardinality are derived to facilitate the 5D BnB search.Experiments on both synthetic data and real images confirm the efficacy of the proposed method.A scene constraint based method is proposed for relative pose estimation between a 2D color camera and a 3D sensor such as a depth camera.The motivation is to use a single pair of images(i.e.one from each camera similar to color camera motion estimation and depth camera motion estimation),and to provide a corresponds-free solution in order to minimize human intervention.To this end,we propose to make use of known geometric constraints from the scene,and formulate relative pose estimation as a 2D-3D registration problem minimizing the geometric errors from scene constraints.In addition,a new single-view 3D reconstruction algorithm is proposed for obtaining initial solutions.The experiments show that the method is both flexible and effective,producing accurate relative pose estimates and high-quality color-depth image registration results.A highly-accurate optical flow estimation algorithm based on piecewise parametric motion model is proposed.A key innovation is that,the proposed algorithm fits a flow field piecewise to a variety of parametric models where the domain of each piece(i.e.,shape,position and size)and its model parameters are determined adaptively,while at the same time maintaining a global inter-piece flow continuity constraint.The novel energy function takes into account both the piecewise constant model assumption and the flow field continuity constraint,enabling the proposed algorithm to effectively handle both homogeneous motions and complex motions.The experiments on three public optical flow benchmarks(KITTI,MPI Sintel,and Middlebury)show that the proposed algorithm achieves top-tier performances.A robust algorithm for optical flow estimation in the presence of transparency or reflection is proposed.It deals with a challenging,frequently encountered,yet not properly investigated problem in two-frame optical flow estimation.That is,the input frames contain two imaging layers – one desired background layer of the scene,and one distracting,possibly moving layer due to transparency or reflection.The proposed robust algorithm performs both optical flow estimation,and image layer separation.It exploits a generalized double-layer brightness consistency constraint connecting these two tasks,and utilizes the priors for both of them.The experimental results confirm the efficacy of the proposed method.To our knowledge,this is the first attempt towards handling generic optical flow fields of two-frame images containing transparency or reflection.