Object-based services with optimization in layered multi-source video delivery

We investigate the authoring, streaming and playback of object-based presentations. We also address optimization problems for adapting multimedia content in a scenario with multiple video sources encoded in multiple scalable layers.; We first present the design and implementation of an MPEG-4 software toolkit. The modular design of the components in the toolkit is an advantage over existing MPEG-4 software as it allows components to be reused separately for developing parts of an MPEG-4 application.; We also implement a new MPEG-4 node and a proposed client-server streaming framework; both were adopted by the MPEG-4 Systems specification. The node allows external applications (PDF reader or web browser) to be embedded with other objects within a scene while the framework supports more sophisticated two-way communication between the client and the server.; We then investigate object-based content authoring issues and present a comparative analysis between MPEG-4 and the Synchronized Multimedia Integration Language (SMIL). Using SMIL, we create a PDA-based application and compare its capability with Macromedia Flash for supporting mobile applications. In view of the several limitations of MPEG-4, we co-developed the eXtensible MPEG-4 Textual Format (XMT), which was adopted by the MPEG-4 Systems specification. The goal of XMT is to improve the ease of content authoring and to support interoperability among existing standards.; In the area of media adaptation, we address the problem of rate scaling of multiple video sources with sequences encoded in multiple layers. The scenario of multisource video streaming evolves naturally from a multi-object streaming application where resources are allocated among the various objects. Our work uses a combinatorial optimization technique---Multiple-Choice Knapsack Problem (MCKP). We develop several heuristics including one suitable for applications where a different priority is assigned to each source. In addition, we design a message passing protocol for synchronizing transmission of state information in a multi-camera video surveillance system and investigate the communications complexity of both centralized and distributed network configurations.; Finally, we introduce the first genetic algorithm for solving MCKP. Our algorithm consists of several components designed utilizing problem-specific knowledge based on the characteristics of MCKP and yields performance improvement over existing heuristics.