Incorporating feature matching and supervoxels into multiframe optical flow

The focus in this dissertation is to estimate the motion of pixels in images. Specifically, we make contributions in the areas of image registration and optical ow. To this end we provide two new frameworks in this thesis --- (1) a relational image model and (2) integration of supervoxel estimation techniques into optical ow.;Our relational image model treats images as point sets in a higher dimensional space obtained by augmenting pixel locations with pixel intensities. Consequently, this treats both pixel locations and pixel attributes (intensity, color, features) on the same footing, which is a major departure from the prevalent functional image model where images are treated as intensity functions of pixel locations. Our work in affine alignment---termed RCA (Relational Color Alignment)---shows how a relational image model is able to overcome the limitations of image derivatives and interpolation in the registration process, and is expedient in estimating both the photometric and geometric transformations.;For the case of estimating the optical ow in video frames, we augment the currently available state-of-the-art objective functions with a sparse correspondence term (obtained by matching edge points) which steers the optical ow in its direction. We extract edges using the Canny edge detector and augment these extracted point locations with their features (intensity or color values). These feature attributed edge points are used to find sparse correspondences using our relational approach, which can be used to enhance the optical ow. Our worktermed as FACT (Feature Attributed Correspondence Tracking) for estimating optical flowshowcases significant performance gains on the challenging MPI Sintel ow dataset.;Finally, in this thesis we tackle the problem of estimating the pixel motion across a sequence of frames. Our work shows that incorporating supervoxels from the video segmentation literature can provide a reasonable initialization which can be a key factor in the performance of locally optimum solvers, specially when the frames undergo a huge deformation. We provide a principled framework to harmoniously integrate any available two-frame optical ow method with any available supervoxel extraction method in an iterative multiframe approach in order to estimate motion spanning over long intermediate video frames.