| While efforts in multimedia information extraction have been traditionally focused on individual analysis processes, this paper presents the integrated analysis mechanism and architecture, which seeks to address many of the traditionally difficult issues from a different viewpoint. Motivated by some important abilities illustrated in human perceptual and cognition behavior, the integrated multimedia analysis views the interpretation of multimedia data as a process of constructing the most coherent and consistent information hierarchy. With effective ways to integrate various types of analysis approaches, knowledge, and interpretation tasks, it is able to overcome many inherent limitations of current systems.; Based on the fundamental analysis mechanism, we also proposed and developed a novel general framework for the integrated analysis, where various analysis processes can enhance and cooperate with each other. The framework does not depend on a particular application or design scenario and, once being supplied with necessary knowledge and configuration information, is able to accomplish tasks essential to the integrated analysis in automatic ways. Such a general framework will not only greatly facilitate the development of integrated analysis systems, but also make them more flexible and extensible than traditional systems.; To demonstrate and evaluate the proposed integrated analysis scheme, we developed the InteMedia system, an integrated audio-visual analysis system consisting of a variety of analysis processes for video cut detection, audio segmentation and classification, face detection and tracking, and person identification. Compared to traditional analysis approaches, the integrated system greatly improved the analysis performance for almost all of the analysis tasks. Information generated based on integrated analysis is much more coherent with the removal of conflicting interpretations among different analysis processes. This system also showed some interesting analysis abilities that are not possessed when performing individual analysis processes independently. |
