Information brokering over heterogeneous digital data: A metadata-based approach

Information overload, arising from different types of heterogeneous digital data readily accessible from millions of repositories, is a critical problem on the Global Information Infrastructure (GII). We present an information brokering approach, architecture and techniques that address issues related to information overload on the GII. The approach spans three levels: representation (structure/format/type) of digital data, information content captured in the data; and the vocabulary underlying the data. Metadata (data/information about data) is used to abstract from heterogeneous representational details and capture information content. Domain specific ontologies are used to represent and interoperate across different vocabularies used to characterize information content. The approach thus suggested induces a metadata-based architecture that enables information brokering at the different levels.; The feasibility of the approach is demonstrated by using a wide variety of metadata to capture information content for textual, image and structured data. These metadata belong to a wide spectrum and may range from metadata independent of the data content to those capturing information content in a application and domain specific manner. This thesis demonstrates how metadata characterizing information in a domain specific manner may enable: (a) media-independent correlation of information across heterogeneous media; and (b) vocabulary-based interoperation of information across different domains.; Example information brokering prototypes based on metadata capturing information content to varying degrees are presented as instantiations to validate the proposed architecture. We also identify the desired ("SEA") properties of an architecture in the presence of information overload, namely, scalability, extensibility and adaptability; and discuss in what measure the prototypes display these properties. The intrinsic trade-off between scalability and extensibility is identified and discussed. Adaptability, a new proposed property, is the ability of an information brokering system to adapt to different vocabularies used to describe similar information content. We show how maximizing scalability leads to issues of adaptability and how terminological relationships across domain specific ontologies characterizing vocabularies may be used to achieve interoperation and increase adaptability.