Feature-based indexing in visual information systems

One underlying characteristic of multimedia information systems is the visual nature of a majority of multimedia data types, making visual information systems an inherent and important aspect of multimedia systems. Visual information systems allow the computer integration, manipulation and presentation of various disparate visual data types, leading to substantial improvements in the quality of the information delivered to the user. The integration of various media types however poses serious problems in providing content-based access to the information content. The problems are further compounded by the enormous differences in individual perception of the contents of a visual information item, the diversity of user information needs, the huge amount of data often involved, and the diversity of visual cues with which visual information can be considered. This dissertation identifies the requirements for content-based indexing and retrieval in visual information systems, and develops some methodologies for robust and efficient indexing in such system.; The thesis starts by identifying the similarities that exist among the various visual data types, and then proposes an indexing framework that allows access to the different data types to be performed in a uniform and consistent manner. The problem of indexing and retrieval in visual information systems is then viewed from two related stand points: image indexing and video indexing. Image indexing treats the visual data as static images and aims at capturing representations that are robust for evaluating the similarity between two images, under various possible changes in view condition. Video indexing uses image indexing as its last step. The initial steps involve video partitioning and the selection or generation of abstractions for the partitioned video. The central themes of the proposed method are the use of a feature-based approach--i.e. indexing and retrieval based on the important features in the visual information items, and the use of the same features uniformly for different visual data types. The basic index features used are the invariant colour ratio features.; In an attempt to address the seemingly diametrical problems of effectiveness and efficiency in video indexing, the thesis introduces the concept of adaptive video indexing. This uses a scene characterisation process to adaptively choose the indexing parameters based on the specific characteristics of the video scene being indexed. The feasibility of the proposed methodologies have been evaluated through experimental investigations on images and video (both for compressed and uncompressed video).