| Understanding the semantic relationships between data objects has been a critical step towards getting useful semantic information for better integration, search and decision-making. This thesis addresses the problem of semantic relationship understanding from two aspects: first, given an ontology schema, an automatic method is proposed to understand the semantic relationships between image objects using the schema as a useful semantic source; second, given a large ontology with both schema and instances, a learning-to-rank based ranking system is developed to identify the most relevant semantic relationships from the ontology according to user preferences.;The first part of this thesis presents an automatic method for understanding and interpreting the semantics of raw web images (represented in pixels, without any annotations). We observe that the relations between objects in an image carry important semantics about the image. To capture and describe such semantics, we propose Object Relation Network (ORN), a graph model representing the most probable meaning of the objects and their relations in an image. Guided and constrained by an ontology, ORN transfers the rich semantics in the ontology to image objects and the relations between them, while maintaining semantic consistency (e.g., a soccer player can kick a soccer ball, but cannot ride it). We present an automatic system which takes a raw image as input and creates an ORN based on image visual appearance and the guide ontology. Our system is evaluated on a dataset containing over 26,000 web images. We demonstrate various useful web applications enabled by ORNs, such as automatic image tagging, automatic image description generation, image search by image, and semantic image clustering.;In the second part of this thesis, a learning-to-rank based ranking system is proposed for mining complex relationships on the Semantic Web. Our objective is to provide an effective ranking method for complex relationship mining, which can 1) automatically personalize ranking results according to user preferences, 2) be continuously improved to more precisely capture user preferences, and 3) hide as many technical details from end users as possible. We observe that a user's opinions on search results carry important information regarding his interests and search intentions. Based on this observation, our system supports each user to give simple feedback about the current search results, and employs a machine-learning based ranking algorithm to learn the user's preferences from his feedback. A personalized ranking function is then generated and used to sort the results of each subsequent query by the user. The user can keep teaching the system his preferences by giving feedback through several iterations until he is satisfied with the search results. Our system is evaluated on a large RDF knowledge base created from Freebase linked-open-data. The experimental results demonstrate the effectiveness of our method compared with the state-of-the-art. |
