Superpixel-based Object Cosegmentation And Search

The visual ability of human can easily perform image segmentation, but this ability is a great challenge for computers. In recent years, with the availability of a large amount of images, it is more reasonable and efficient to jointly segment multiple images rather than segmenting each image independently, because images usually contain common object or similar objects of the same category. These approaches are termed as cosegmentation. Based on the extensive review and research on the state-of-the-art cosegmentation methods, this thesis focuses on the study of superpixel-based object cosegmentation, high dimensional descriptor representation and segmentation energy model with its optimization method. Furthermore, it also applies the idea of cosegmentation for simultaneous object search and segmentation. The thesis includes several fields and contributions as follows,1. A new cosegmentation algorithm based on the object discovery scheme and the tree structural constraint is proposed to overcome the foreground/background ambiguity caused by the similar backgrounds in images and the limitation that most Markov random field based cosegmentation methods are designed for small dataset (e.g., image pair). The novel object discovery scheme introduces the awareness information of the superpixel-saliency prior and the superpixel-repeatness measure to leverage the prior information to solve the foreground/background ambiguity. Meanwhile, the superpixel-based segmentation problem is formulated as a combinatorial optimization problem and is solved by a tree-constrained optimization method. This optimization method not only reduces the computational cost but also makes the final segmentation result more precise and complete.2. An automatic learning method is utilized to train a classifier from the training data generated by the object discovery process, thus making our approach scalable to larger datasets. Furthermore, a structural forest model and its optimization method are proposed to extend the tree-constrained single-object segmentation to multi-object segmentation and solve the limitation that existing superpixel-based cosegmentation methods are not so effective for multi-object scenarios (e.g., sporting events).3. A novel and unified awareness-based segmentation model is proposed to overcome the limitations that some step-by-step cosegmentation methods are sensitive to initializations and they just utilize the prior information such as common object prior and saliency prior at the initialization stage. The awareness information that is based on area-saliency, area-repeatness and area spatial layout is introduced as a global constraint into the energy model. Meanwhile, an awareness-based structural forest framework is utilized to optimize this energy model and make full advantage of the awareness information in object cosegmentation.4. Inspired by the idea of interactive cosegmentation, a novel simultaneous visual object search and segmentation model is proposed to perform a cooperative object search and segmentation in Internet images. In this model, the seed superpixel links the search and segmentation processes and makes them work in a mutually rewarding way. The seed superpixel serves as the structural constraint to guide the segmentation process, and it is then updated by a validation process based on the segmentation results. The final segmentation result can be regarded as the outcome of object search and segmentation.