Optical Flow Estimation On Monocular Video

The concept of optical flow arises from studies of biological visual systems, which describes the apparent motion observed in a sequence of images. A displacement field can approximately characterise an optical flow field, which takes into account the projection from 3D to 2D image plane. Studies of visual perception reveal that human beings recognize their environment from motion information. The automatic navigation experiments also prove that the optical flow information as a guide is better than traditional navigation messages. Optical flow estimation has been one of important tasks in computer vision and related research areas, since objects motion is the main body of visual information. Optical flow has a wide range of applications such as object tracking, robotics, human–machine interaction, driver assistance systems, video compression, super-resolution, and dynamic texture analysis.The most widespread approach to define and calculate optical flow field is based on brightness constancy assumption (BCM). However, BCM is only valid for Lambertian surfaces at constant illumination and there is an aperture problem, which limits applications of optical flow technology. Focusing on specific applications, improved optical flow models are presented in order to increase accuracy and stability of optical flow computation.This dissertation involves optical flow estimation for color video, optical flow calculation in variable illumination and large displacement optical flow. The main research and contributions are listed as follows:1. In order to improve the stability of optical flow vectors, a perception-based color space with linear normalization is proposed by using color information, which takes into account color perception properties of human eyes. The difference vectors between color pixels in this new color space are unchanged by reillumination, and the 2 norm of a difference vector matches the perceptual distance between two colors. The linear normalization greatly reduces the correlation between color components, and this normalized color space is suitable for optical flow estimation. The ratio color space based on linear normalization is presented to further enhance the robustness of color space to changes in viewing direction, object geometry, highlights and direction of the illumination.2. Classic optical flow estimation only adopts brightness, gray or the other monochrome component without fully using rich color information. Optical flow field is the projection from 3D motion to 2D plane which belongs to ill-posed problem, so it needs other constraints to obtain a unique solution. It would cause a large number of optical flow errors if the auxiliary conditions are inappropriate. Because of some color components are more stable than brightness in illumination changes, this color information can be introduced into optical flow calculation for improving estimation quality. The color overdetermined equations of optical flow with extended constraints are proposed, which combine the reconstructed color space with small smooth region. Experiment results show that the proposed method is superior to the similar optical flow calculating methods in natural videos.3. Optical flow vectors are tend to be over-smoothing on motion boundary that leads to more outliers because of the existing limitation of numerical technologies. For this reason, the improved automatic color segmentation based on cell growth is presented. With accurate and consistent motion boundary coming from segmentation, optical flow errors decrease and outliers reduce by filtering in different movement regions.4. BCM is unstable in variable illumination, so there have been various improved optical flow models for mutative lights. General dynamic image model (GDIM) has large influence and wide applicability in these models. Based on GDIM, parameters of GDIM are explained and amended in Clifford algebra. And then, an optical flow model based on illumination radiation is proposed which bases on Clifford algebra and post-experiments data arising from principle of generating movement vision and illumination radiation theory. Following, a total variation algorithm is described. Last, experimental results show that the proposed model can achieve accurate and consistent optical flow fields in different illumination for general purpose.5. Large displacement means high speed motions in videos. Traditional linearization in optical flow models is only valid for velocities of small magnitude which limits in one pixel. In the case of large motion vectors that arise in most real-world applications, total variation computation based on duality warping which integrates color SIFT matching is proposed. After obtaining preliminary large displacement, the optical flow vectors are adjusted with guiding in color SIFT matching based on Hausdorff distance in multi-dimensional point sets. The proposed method preserves details and enhances the quality of estimation for large displacement.