Image Patch-based Dense Correspondence Methods And Their Applications

Visual object tracking is a hot topic in the computer vision and pattern recognition community. It has many promising applications including human computer interaction, intelligent surveillance, and medical image processing, etc. In general, a typical visual object tracking system is composed of four modules:object initialization, appearance modeling, motion estimation, and object localization. Among these modules, appearance modeling is one of the most critical prerequisites for successful visual tracking. Designing an effective appearance model, however, is a challenging task due to appearance variations caused by background clutters, object deformation, partial occlusions, and illumination changes, etc. To deal with these challenges, in this thesis, we focus on how to design a robust appreance model for visual tracking using different visual representations and/or statistical modeling techniques.This paper presents a novel patch-based match and fusion algorithm by taking account of moving scene in a multiple exposure image sequence using optimization. A uniform iterative approach is developed to match and find the corresponding patches in different exposure images, which are then fused in each iteration. Our approach does not need to align the input multiple exposure images before the fusion process. Considering that the pixel values are affected by various exposure times, we design a new patch-based energy function that will be optimized to improve the matching accuracy. An efficient patch-based exposure fusion approach using the random walker algorithm is developed to preserve the moving objects from the input multiple exposure images. To the best of our knowledge, our algorithm is the first patch-based exposure fusion work to preserve the moving objects of dynamic scenes that does not need the registration process of different exposure images. Experimental results of moving scenes demonstrate that our algorithm achieves visually pleasing fusion results without ghosting artifacts, while the results produced by the state-of-the-art exposure fusion and tone mapping algorithms exhibit different levels of ghosting artifacts.A new method is presented to compute the dense correspondences between two images by using the energy optimization and the structured patches. Many transformation and deformation cues such as color, scale and rotation should be considered when we finding dense correspondences between images. However, most existing methods only consider part of these transformations, which will introduce the uncorrect correspondence results. In terms of the property of the sparse feature and the principle that nearest sub-scenes and neighbors are much more similar, we design a new energy optimization to guide the dense matching process and find the reliable correspondences. The sparse features are also employed to design a new structure to describe the patches. Both transformation and deformation with the structured patches are considered and incorporated into an energy optimization framework. Thus, our algorithm can match the objects robustly in complicated scenes. Finally, a local refinement technique is proposed to solve the perturbation of the matched patches. Experimental results demonstrate that our method outperforms the state-of-the-art matching algorithms.Multi-videos object segmentation and reconstruction is to segment the foreground object in the videos, as well as to reconstruct a 3D model of this foreground object. We present a new unsupervised method to simultaneously segment and reconstruct the foreground object from multi-videos using patch-based optimization framework. The inter-video and intra-video foreground object coherence are considered, i.e., we design a dense correspondence algorithm to match different videos (inter-), and compute correspondences between pairs of subsequent frames (intra-). Since the segmentation and reconstruction of subsequent frames will be initialized by the former foreground estimate, we propose a model updating technique to refine the foreground object segmentations. Experimental results on several open datasets (e.g. FBMS-59[151], Youtube-Objects [152]) demonstrate the effectiveness of our method.