VIVIM: Video indexing for visual information management

The visual form of information provides a primary means of communication. Recently, video has become one of the largest sources of visual information. The last few years have seen a remarkable growth in the acquisition and storage of video information in the digital form. But digitally acquired video poses new problems for its management and will require new dimensions of engineering for its effective use.; The progress made on visual information systems can be grouped into three distinct categories. The first category, known as manipulation and composition, consist of efforts aimed at using generation of new information from previously existing. This includes authoring and editing systems. The second category of research is directed towards information compression issues. The third category, known as content characterization and indexing, of research has concentrated on providing means for organization and retrieval of visual information. For video data this is accomplished by segmenting the video into individual clips and then characterizing them for their content. The scope of this work is limited to this last category of research because content characterization and indexing are fundamental to visual information systems.; Video segmentation comprised the first task in the work. An automatic video segmentation algorithm was created for robust partitioning of video into its generic segments. Our goal was to fulfill a partitioning criterion for video which will permit determination of scene changes. A unique compressed information processing algorithm based on local and global spatial information was developed in this regard.; Video indexing comprised the second task. Although video indices can be generated using both visual and auditory information, past efforts have only concentrated on use of visual information. In our work we have used both visual and auditory information in the indexing process. A unique decision tree based algorithm was proposed for generating camera motion classes using motion vector information available in MPEG encoding of video. A compressed audio domain information manipulation was also developed and utilized for generating audio classes and subsequently identifying the speaker in video segments. The experimental results were validated using nearly 25 minutes of commercial video.