Research On Weakly Supervised Automatic Image Region Annotation

In this thesis, we concentrate on a challenging problem---weakly supervised image region annotation, which are trained on images with weakly supervised information, i.e., image-level labels. Image region annotation bridges the gap between high-level semantics and low-level pixel representation of images in machine.Traditional image region annotation needs a large number of images with region-level labels. However, labeling pixel with semantic label manually is time consuming and image-level labels, which are ambiguous and difficult for training, are easy to access. The explosive generation of web data provides a large amount of raw data for weakly supervised problems. Since the rare research on weakly supervised image region annotation, the contributions of our work mainly focus on following aspects:1、We gave a framework introduction on weakly supervised machine learning. Since the ambiguity of image-level labels, context information id needed to reduce the ambiguous degree of image level labels, which is named collectively annotation. We introduced KNN and sparsity to find contextual information of the target. We also introduced how to make use of contextual information to construct semantic graph and how to propagated label information on the graph. We introduced two graph propagation methods, Markov Random Walks and Spectral Clustering.2、We proposed how to construct context based on discriminative semantics. We noticed that the main difficulty of weakly supervised image annotation is how to find valuable information for effectively collective annotation and traditional discovery of contextual information based only on similarity and relevance can lead to context homogenization problem. In context homogenization situation, target superpixel and contextual superpixels have similar visual appearance near duplicate image-level labels. To overcome the context homogenization problem and make full use of semantic and visual information of images, we consider not only visual and semantic relevance, but also the semantic distinction between neighbor superpixels and the target superpixel in the process of affinity graph construction. We proposed how to construct discriminative lysemantic graph (DSG) based on discriminative semantics.3、We proposed how to propagate label information via the constructed discriminatively semantic graph. We constructed two types of graphs, the inter-image graph (DSG) make full use of contextual information to get the label distribution. The final label assignment is solved by energy minimization on an intra-image contextual graph via Graph Cuts algorithm, which is constructed on a single image, considering visual similarity and semantic relevance of superpixels.Our experiment was conducted on datasets MSRC-21 and PASCAL VOC 2007. We used per-class accuracy and average per-class accuracy as the evaluation metric. Compared with state-of-the-art methods, our framework achieved satisfying performance in per-class accuracy and average per-class accuracy.