Research On The Theories And Methods Of High Efficiency Video Coding And Its Perceptual Quality Assessment

As entering into the information age, the multimedia data represented by the digital images and videos grow dramatically, and become the main carrier of the information.Although the digital video is popular due to the intuitive feature, its huge amount of information results in very high demand for the storage capability and the bandwidth of network, which hinders the acquisition and use of the video data. Therefore, it is of great importance in developing the high efficiency video coding technology for the visual communication and multimedia related industries, which could store and transfer the compressed video data. The main target of video coding is to remove various redundancy information from the video data. Firstly, from the objective perspective, the spatial and temporal neighboring pixels are highly correlated, which need accurate prediction technology to reduce these spatial and temporal redundancies. Limited by intra/inter prediction accuracy, the performance of existing video coding standards still can’t satisfy our requirement for transferring the high definition, ultra high definition and multi-view videos. It is urgent to develop more accurate intra/inter prediction technology in the field of multimedia communication. Secondly, from the subjective perspective, the human vision system presents different sensitivities for the distortion under different visual contents. It is the key to reducing the perceptual redundancy and improving the video compression efficiency by developing accurate perceptual quality assessment metric, which could guide the mode decision and parameter selection. Based on above considerations,this paper carries out the research on high efficiency video coding and its perceptual quality assessment.Our work mainly starts from two aspects, i.e., improving the video compression ratio and the subjective quality of the reconstructed video. As developing some high efficiency coding tools, we further design the efficient perceptual image quality assessment methods based on the property of human vision system, which are used to optimize the postprocessing parameter for the reconstructed video. The detailed research contents and innovations have been listed in the following:1. High performance intra prediction methodThe correlation of neighboring pixels is closely associated with their spatial distances and the local texture direction. The traditional directional intra prediction(DIP)ignores the distance variation between current pixel and its references, and assigns the same prediction value to the pixels in the major prediction direction. As the spatial distance of neighboring pixels increasing, the prediction accuracy gradually drops. To address this problem, we build a prediction distance model which considers both the major direction in a block and the spatial distance between neighboring pixels. Then, a mode dependent adaptive down-sampling and interpolation prediction method is proposed based on this prediction distance model. The proposed method could efficiently reduce the prediction distance between neighboring pixels and increase the intra prediction accuracy,which provides the foundation for further improving the overall compression efficiency of the video coding framework.2. High performance loop filter methodThe loop filter is an important part of the video coding framework. Its main task is to improve the quality of a reconstructed image and provide accurate reference for the following inter prediction. Traditional deblocking loop filter(DLF) ignores the differences between different image contents and uses fixed low-pass filter to suppress the blocking artifact. For the local region with complex scenes, this method would loss rich of detail information. To address this problem, we propose a mode dependent adaptive loop filter. The proposed method utilizes the intra prediction mode to describe the local texture information. Then, a series of filter parameters would be designed for different local textures by online training, which adjusts the filter for different image contents. In this way, we can further preserve the image details while reducing the blockiness.3. High efficiency non-parametric perceptual quality assessmentThe efficient perceptual image quality assessment method is an important basis for evaluating, monitoring and enhancing the quality of the video compression service. Since the user can only access the edited video and image contents, a no-reference image quality assessment(NR-IQA) metric which doesn’t need information from the original reference image is quite desirable in practical application. Here, we propose a multi-channel feature fusion and label transfer based NR-IQA algorithm. For the image description and quality prediction, the proposed method could efficiently simulate the hierarchical structure and visual memory retrieval process of human vision system. Since our label transfer model doesn’t need to learn the parameters, we can implement the image quality assessment with low computational load.4. High performance parametric perceptual quality assessmentFor the NR-IQA, the non-parametric prediction model possesses the superiority in low complexity. However, due to lack of the tuning from supervised information, its prediction accuracy still needs to be improved. Thus, we propose a multi-domain structural information and piecewise regression based parametric NR-IQA method. On one hand,we combine the structural information from both the spatial and frequency domains to capture the natural scene statistics variation of an image. On the other hand, we learn the specific model parameters for each test image by online training. The proposed method could efficiently capture the local distribution of feature space, which delivers more accurate quality prediction results.5. Perception-driven image deblocking filterThe deblocking filter is critical in improving the perceptual quality of the compressed videos. Traditional DLF reduces the blockiness of a reconstructed image with pre-defined low-pass filter. Due to lack of efficient feedback mechanism, both the video sender and receiver can’t determine the perceptual quality of the filtered image and implement an efficient enhancement. To guarantee that the image enhancement could produce satisfying perceptual quality, we propose a perception-driven image deblocking filter method. The proposed method utilizes a parametric shape adaptive DCT(SA-DCT)filter to enhance the image quality. Meanwhile, we employ our proposed NR-IQA metrics to monitor the perceptual quality variation under different filter parameters and report it to the SA-DCT filter. Based on the image quality feedback, we can solve the optimal filter parameters in terms of NR-IQA metric. Experimental results show that the proposed method could efficiently improve the subjective quality of the compressed videos.