Digital multimedia library indexing and retrieval

From Ptolemy's library at Alexandria to the various kinds of modern libraries, people have been looking for ways to store, organize, and retrieve recorded knowledge and information. Today, multimedia and networking technologies have brought more and more colorful informations into our daily life. Meanwhile, the outburst of extremely large amount of multimedia information, which includes an integration of a variety of heterogeneous data: text, voice, audio, graphics, images, animations, and video, demands new methods to maintain and retrieve the information efficiently.; This thesis addresses the key technical problems arising in multimedia data management and manipulation by proposing new methodologies and algorithms for automatic video indexing and retrieval. The design of multimedia data modeling, operation, organization, and retrieval is discussed in detail with emphasis on visual (video and image) data retrieval, the most challenging problem of multimedia data management. New techniques for a complete framework of automatic video indexing system are developed to analyze and manipulate the visual content of digital video and image for digital multimedia library applications. This thesis proposes novel perspective on characterizing visual content of video data for video segmentation and searching.; A hierarchical multi-resolution video segmentation scheme for high precision video shot transition detection is presented. Unlike previous approaches which mainly relied on identical feature and algorithm to detect all kinds of shot transitions but often failed to identify dissolve and wipe transitions, the proposed new approach can detect as well as identify all four kinds of shot transitions hierarchically. By studying the production aspect of video and formulating the shot transitions accordingly, robust features are developed to capture the changing statistics of each of the four kinds of shot transitions, and consequently identify each of them successfully. The proposed hierarchical video segmentation scheme is then applied to a sample application, automatic video editing.; Subject-based retrieval for automatic real world video database searching is proposed. The majority of previous work on content-based retrieval has been concentrated on similarity-based retrieval for web-based image search engine and scientific imaginary. However, film- and TV program-oriented video often does not have repeated views of subjects which make it hard to conduct similarity-based retrieval. This thesis proposes a new technique for automatic video database retrieval that is beyond the application of web image and scientific image search. The system analyzes the visual content of video keyframe images off-line and outputs a series of descriptive symbols which contains the visual content and spatial information of scenic objects. During the retrieval stage, user can retrieval images and videos of her interest by specifying query subject using the predefined subject lexicon. The system is capable of retrieving specific subjects as well as general categorization subjects. Futhermore, the methodology of subject-based retrieval can also provide a way for multimedia data integration for the purpose of multimedia database retrieval.; LOST language is proposed for efficient query formulation. It inherits and incorporates the advantages of string, graph, and logic languages to help the system retrieve user query in a much simpler and more effective way. In addition, it provides the capability of retrieving scenes of interests with logic, spatial, and temporal specification. The indexing technique presented in this thesis is unique in that it is driven by video applications in the real world domain—film- and television program-oriented digital video libraries.