Robust visual motion analysis: Piecewise-smooth optical flow and motion-based detection and tracking

This thesis describes new approaches to optical flow estimation and motion-based detection and tracking. Statistical methods, particularly outlier rejection, error analysis and Bayesian inference, are extensively exploited in our study and are shown to be crucial to the robust analysis of visual motion.; We first take a local approach to optical flow estimation, that is, finding the most representative flow vector for each small image region. We recast the popular gradient-based method as a two-stage regression problem and apply adaptive robust estimators to both stages. The estimators are adaptive in that, their complexity increases with the amount of outlier contamination. To characterize the spatially varying uncertainty, we perform error analysis systematically through covariance propagation.; Pointing out the limitations of local and gradient-based methods, we further propose a matching-based global optimization technique. The problem is formulated as maximizing the a posteriori probability of the optical flow given three image frames. Using a Markov random field flow model and robust statistics, the formulation is reduced to minimizing a global energy function, which we carefully designed to allow outliers, occlusions and local adaptivity. A three-step graduated solution method is developed for the resulting large-scale nonconvex optimization problem. It takes advantages of various popular techniques and achieves high efficiency and accuracy. The performance is demonstrated through experiments on both synthetic and real data and comparison with competing techniques.; The last part of the thesis describes a motion-hased detection and tracking system designed for an airborne visual surveillance application, in which challenges arise from the small target size, low image quality, substantial camera wobbling and background clutter. The system has two components: a detector identifying suspicious objects by the statistical difference between their motion and the background motion, and a Kalman filter tracking the dynamic behavior of objects in order to detect real targets and update their states. Both components operate in a Bayesian mode and each benefits from the other's accuracy. The system exhibits excellent performance in experiments. In an 1800-frame real video, it produces no false detections and tracks the true target since the second frame, with average position error below 1 pixel.