Error Resilient Video Coding over Error Prone Networks

Video communication and other web-based video applications become popular in recent years. However, the transmission of the compressed video bit stream often suffers from imperfection of the communication channel, like path loss, multipath fading, co-channel interference, congestion, etc. Error resilient video coding techniques need to be employed to mitigate the channel errors, which include error concealment in the decoder, forward error correction in the encoder and joint encoder-decoder error control techniques.;In this thesis, techniques for efficient error resilient video coding are investigated. Three parts of the work are discussed in this thesis.;In the first part, decoder based error concealment methods are discussed. An adaptive partition size (APS) temporal error concealment method is developed for H.264. We propose to use Weighted Double-Sided External Boundary Matching Error (WDS-EBME) to jointly measure the inter-MB boundary discontinuity, inter-partition boundary discontinuity and intra-partition block artifacts in the corrupted MB. By minimizing the WDS-EBME value of each partition, the best motion vectors of each candidate partition mode can be estimated, overall WDS-EBME of the MB concealed by each partition mode can then be evaluated and the best partition mode for the corrupted Macroblocks (MB) will be determined as the one with the smallest overall WDS-EBME. We also propose a progressive concealment order for the 4x4 partition mode.;The second part of this thesis investigates encoder based error control techniques. Firstly, a VLC/FLC data partitioning method is proposed for MPEG-4. It disables intra AC prediction and groups appropriate fixed length coded (FLC) syntaxes in a video packet (or slice) together to form a new partition. With intra AC prediction disabled, errors occurring in these FLC syntaxes will not cause spatial error propagation. It essentially classifies the syntaxes into two categories according to whether that syntax will cause spatial error propagation when an error occurs. Secondly, a redundant macroblock strategy is proposed for H.264. MB Differential Mean Square Error (DMSE) is employed to evaluate the error sensitivity of MBs. The most sensitive MBs are transmitted separately in additional slices while coarsely quantized copies of the MBs are placed in the original slice. When working with chessboard style Flexible Macroblock reordering (FMO) and fixed length slice mode (FMO-slicing), the scheme performs well against packet loss errors with acceptable overhead and it is highly compatible with original H.264 bitstream. Thirdly, a joint optimal bit allocation and rate control scheme is proposed for H.264 with redundant slice. The optimum ratio between each primary and redundant picture pair is analytically deduced. Rate function and distortion model for both representations are developed, and a simple close-form solution is provided to achieve joint optimum bit allocation.;The last part of the thesis concerns the joint encoder-decoder error control method. A joint temporal error control method is proposed for H.264. It combines RDO-based macroblock (MB) classification at the encoder and adaptive partition size error concealment at the decoder. The encoder classifies the MBs by evaluating the sensitivity of the MBs as the RD cost between the concealment error and the bits needed for the additional motion information. Additional motion information such as the original motion vector or motion vector index can be transmitted for the error sensitive MBs. The decoder utilizes the additional motion information if any of these MBs get lost. Non-sensitive MBs and blocks are concealed by the APS method.