Low-complexity mode selection for rate-distortion optimal video coding

The primary objective of this thesis is to provide a low-complexity rate-distortion optimal coding mode selection method in digital video encoding. To achieve optimal compression efficiency in the rate-distortion framework with low computational complexity, we first propose a rate-distortion model and then apply it to the coding mode selection problem. The computational complexity of the proposed method is very low compared to overall encoder complexity because the proposed method uses simple image properties such as variance that can be obtained easily. Also, the proposed method gives significant PSNR gains over the mode selection scheme used in TM5 for MPEG-2 because the rate-distortion model considers rate constraints of each mode as well as distortion. We extend the model-based mode selection approach to motion vector selection for further improvement of the coding efficiency.; In addition to our theoretical work, we present practical solutions to real-time implementation of encoder modules including our proposed mode selection method on digital signal processors. First, we investigate the features provided by most of the recent digital signal processors, for example, hierarchical memory structure and efficient data transfer between on-chip and off-chip memory, and then present practical approaches for real-time implementation of a video encoder system with efficient use of the features.