Image Understanding Via Embedding Structured Feature Representation

With the development of technologies on Internet and Mobile Internet, millions of images are uploaded to the Internet which is growing exponentially every day. The age of big data is coming. How to mine the hidden value of these data is a hot research topic, as it would be involved in a lot of technologies. One of the most important steps is image understanding. The traditional methods on image understanding are based the Bag of word model. The low-level features of images are extracted from the images, and then a dictionary is constructed based on these features. The representation histogram of each image is developed by mapping the features samples. Although the bag of word model has been achieved some expected results on image understanding e.g. image classification and image retrieval, this model loss the structure information of images which would decrease the discrimination and robustness of image features. Different from the traditional methods, in this paper, we propose a novel feature representation method which can embed the structure information into the feature representation to improve the discrimination and robustness of the images features. Specifically, there exist three kinds of structure information, which are applied for image retrieval, image classification, and image annotation.Firstly, different from the traditional methods on sketch classification and retrieval which directly use the low-level features, we propose to embed the symmetry structure of objects in the image into the low-level features. With the embedded information, the novel feature representation is more discriminative and robust. In the experiment, we use such feature representation for sketch classification and retrieval. The experiment results show that our proposed representation can improve the performance of accuracy, which demonstrate the effectiveness of our method.Secondly, on multi-attribute based image retrieval, we propose to embed the structure of attributes co-occurrence and exclusive into the middle level feature representations instead of only using the co-occurrence between attributes. Firstly, we build the feature representations of images with the structure of attributes. Then, a novel framework based on the feature reconstruction is proposed to preserve the structure of representation when we compute the distance between the query and images. Experimental results show that this framework can decrease the ambiguity of queries and improve the performance of image retrieval.Thirdly, the traditional methods on weakly supervised image annotation would generate the ambiguous feature representation for each image, especially there exist several objects in the image. As the above-mentioned facts, bag of words model loss the structure correlation between different objects in the image which can cause the inaccuracy of image annotation. In this paper, we propose to embed the high level semantic label information into the feature representation to improve the discrimination of the original feature. The experimental results show that our method can improve the performance of image annotation, which further demonstrate the effectiveness of our novel feature representation.We propose a novel feature representation by embedding the structure information into the original feature on image understanding. Considering different scenarios need distinct structure information, we propose to use the symmetry structure of objects in the low-level features, to use the structure information between attributes and the semantic structure between labels. The experimental results on different applications demonstrate that our proposed features are discrimination and robustness. Furthermore, our proposed feature representation with the embedding structure information could also be beneficial for the advancement of computer vision.