Coloring Bag Of Words Based Image Representations

Object recognition is a basic problem in computer vision, generating discriminative image representation is an important way to solve this problem. Bag-of-Words (BOW) is an outstanding image representation method, first, local features are extracted and then used to generate visual words, finally, the occurrence frequencies of all the visual words are used as the image representation. However, BOW has some shortcomings, for exam-ple, BOW only considers the shape features and spatial information is not considered. In order to overcome the shortcomings of the model, in this dissertation, color is utilized to identify the object area, and fuse color and shape features to generate a discriminative image representation based on BOW. Our contributions are:(1) A color combination based image representation is presented. The shape feature is weighted by the color feature. The discriminative colors are estimated and com-bined to assign a unique and high weight to the object patches. An object is consid-ered as a group of patches with different discriminative colors, due to the different occurrence frequencies of different discriminative colors, the object patches can-not be assigned the same weight. The optimal discriminative colors are obtained by using the inter-class and intra-class color similarities, and then these discrimi-native colors are combined to assign the weights for the patches. Experiments on benchmark images confirm the advantage of the proposed method.(2) A contextual color attention map is proposed to represent images. The colors are divided into two types:strong colors and weak colors, and the patches with strong colors and weak colors are called strong patches and weak patches, respectively. Suppose that the strong patches are the object patches and the weak patches are the background patches. Thus, the object patches with weak colors are mistaken for the background patches. To identify these false weak patches, the relationship between strong and weak patches are utilized to compute the contextual color attention, and the false weak patches are obtained by using the optimal thresholds. Extensive experiments show that our contextual color attention map based image presentation outperforms top-down color attention (CA) based image representation.(3) A Component Pyramid Matching (CPM) method is proposed for image representa-tion. CPM partitions an image into different levels, the background and foreground in each level represent different components, and the foreground components corre-spond to some parts of the objects. The final representation of an image is obtained by concatenating the representation of each component. CPM captures the spatial information of the images, so it can obtain satisfactory results in some color related image sets.(4) A hierarchical mid-level features mining method is proposed for image representa-tion. Our method uses the discriminative colors to partition an image into different levels, the patches in different levels are considered in the same sub-category, and then an effective mid-level feature mining method is proposed to discover the re-lationship between the local features, the mined patterns from these sub-categories are then used for the final image representation instead of visual words. Experi-ments on benchmark images confirm the advantage of the proposed method.(5) A multi-image matching (MIM) based image representation is proposed. To match the images accurately, a graph is constructed for each image, in which each n-ode represents a patch and edges are added between neighboring nodes. A seed-expansion strategy is adopted to match the images, the discriminative colors are important to obtain the seeds. The matched sub-graphs in different images repre-sent the same object. The match-sets are finally used for the image representation. Extensive experiments show the advantages of the method.Color combination and contextual color attention map based methods utilize the color to weight the local shape features, while color is utilized to partition an image into different levels in CPM and mid-level features mining methods so as to give spatial infor-mation for image representation. MIM based method utilizes the discriminative colors to find the seeds and it can obtain the best classification accuracies among these proposed methods.