Research On Image Correlation Analysis, Exploration, And Its Applications

In recent years, the widespread of photographing devices, such as digital cameras, cel-lar phones, etc., allows people to capture images in an increasingly convenient way. The amount of image resources is further compounded by the rapid development of computer networks and storage devices. Therefore, images today are often presented collectively rather than singly, which makes the intelligent analysis and processing of multiple images jointly a critical topic in the computer vision and multimedia commu-nities.Images within an image set usually correlate with each other. The correlations, if quantized correctly, can be greatly beneficial for understanding and processing the im-age set. Following this idea, in this dissertation, we focus on exploring and leveraging such correlations to facilitate several research tasks, including image recognition, com-mon visual pattern discovery, binary segmentation of an image collection and image co-matting, etc., demonstrating how the mutual associations among multiple images can be mined and exploited to improve the performance when handling a given image set in different task settings. More specifically, our work mainly includes the following four aspects:(1) Feature correlation integrated Bag-of-Words model for image recognition from a given image database. We improve the conventional Bag-of-Words model for image recognition by exploring feature correspondences among database images of the same object. Once established, the correspondence relationships are em-ployed to refine the feature set of each training image, as well as utilized as constraints to guide the training phase. It leads to a more accurate model, which further augments the performance of image recognition.(2) Common visual pattern discovery via nonlinear mean shift clustering. It aim-s to automatically identify common objects from a pair of given images, which entails recovering and grouping correct correspondences of local features. We in-troduce a mean shift based approach to carry out the task. An similarity transfor-mation space is firstly constructed, with each transformation estimated explicitly linking a pair of initially appearance-matched local features. This is followed by a mean shift clustering stage in the space to group closely situated similar transformations. Joining regions associated with the transformations in the same group together within each input image forms two large regions that share sim-ilar geometric configuration, which naturally leads to a common visual pattern. The proposed algorithm can be directly used to establish the association between images and performs well when applied to the task of image retrieval in experi-ment.(3) Example-guided image collection segmentation. We propose an example-based semi-automatic image collection segmentation framework. It starts with select-ing few sample images from the given image collection and deliver them to users for hand-segmentation. Each remaining image is then automatically cutout by learning from the segmentation information of samples and followed by rectifica-tion process with user assistant if necessary. Through exploiting the associations among the sample and target images, it achieves better performance, comparing with co-segmentation techniques, and accurate object cutout with surprisingly few user interactions.(4) Confidence-driven image co-matting. We present the concept of image co-matting, aiming to simultaneously extract alpha mattes in multiple images that contain slightly-deformed instances of the same foreground object against different back-grounds. The task is achieved by first aligning the foreground object instances using appearance and geometric features, then applying a global optimization on all input images to jointly improve their alpha mattes, which allows high con-fidence local regions to guide their corresponding low confidence ones in other images to achieve more accurate mattes all together. The matting confidence of each image is estimated with a multiple feature based learnt metric trained on a specially designed dataset. Experimental results show that the co-matting framework can achieve noticeably higher quality results on an image stack than applying state-of-the-art single image matting techniques individually on each image, providing a good premise for future studies.