| 3D model based coding is an object oriented video coding scheme with the potential to compress video sequences containing human subjects at bit rates under 16 kbits/s. In this dissertation, details of this scheme are presented, with results from a software implementation that runs on a PC, rendering sequences of spatial resolution 352 x 288, at 25 frames/s.; The coding scheme can be divided into two major components, the analyzer and the synthesizer. A 3D facial model is the most important component of both these units. The model adopted for this work is described in detail from geometric and functional perspectives. Details for implementing a synthesizer to produce realistic sequences using this model are presented. A low cost approach is presented for image mapping that can easily be integrated with the synthesizer. This enables the use of a single generic facial model, thereby avoiding the need for special 3D digitizers for mapping the identity of the subject in the sequence or detailed models for each individual appearing in such sequences, with little overhead added to the compressed bitstream. Head movements and facial expressions can be synthesized, by globally transforming the image mapped generic facial model, and actuating the appropriate muscles and/or the joint that are simulated in the model, respectively. Strategies for analyzing sequences with a motion model incorporating both rigid body movements of the face in the sequence, and local deformations due to facial expressions are also discussed. The description of both the synthesis and the analysis procedures incorporate perspective projection, thus making them close approximations to the natural imaging process.; A procedure to achieve automated lip synchronization of the subject in the synthesized frames with the associated speech, is described. Extending the synthesizer for creating sequences with multiple human subjects is presented with simulation results. Adapting this scheme for creating applications in a networked environment is also discussed. |
