Beyond HEVC: Exploring new frameworks for video coding

Current prevalent video coder like HEVC works on a block based hybrid coding scheme with motion estimation by finding a prediction of current block from one or multiple best matching candidates using a fixed weight, and represent prediction residual using fixed orthonormal transform. In my dissertation, I investigate two different frameworks tackling two intrinsic constraints of such coder.;In the first part of this dissertation, I present an adaptive two-stage framework with variable block sizes. The first-stage presents a generalized motion estimation scheme by predicting each image block with a sparse linear combination of atoms in a redundant self-adaptive dictionary consisting spatial-temporal prediction candidates. The prediction residual is coded in a second-stage using an adaptively orthonormalized DCT basis. I also propose an improved Orthogonal Matching Pursuit solver with embedded orthonormalization for the first stage, which iteratively finds the best atom with the largest decay in the rate-distortion curve. The switching point from the first-stage to the second-stage is determined through an RD-aware adaptive switch in a per-block basis. A fast algorithm for determining the quad-tree partitioning structure is also presented to extend the proposed framework for variable block sizes. A context-adaptive binary arithmetic code is designed for coding the various symbols of this framework. The proposed coder has shown competitive RD performance as HEVC and in some cases, achieves better performance than HEVC.;In the second part of this dissertation, I propose a tree-structured object based video coding framework. The framework relaxes the need of performing motion estimation for every image block of a video frame, by constructing a tree-structured parametric motion representation bounded by coherently moving objects via solving an L1-norm minimization problem between the motion-compensated reference frame and the current frame. The root node of this motion representation rectifies the global motion that applies to the whole video frame, whereas the leaf nodes describe the residual coherent motion of objects moving differently than the global motion. The resultant predicted frame consists of the reference frame pixels warped by the global motion, plus additional warping of the regions described by the lower layers. The sub-frame object regions are derived from the previously coded frames and therefore need not be transmitted as side information. The residual frame is coded as HEVC intra frame and the motion parameters are coded losslessly by range coder. In our preliminary study, the proposed framework has shown competitive RD performance as HEVC, and for some sequences with isolated moving objects, the proposed coder can achieve significant gain over HEVC.