Research On H.264 Based Stereoscopic Video Coding

As the development of computer, communication and network technology, the application prospect of stereoscopic video is more and more broad. Its application includes 3D video conference, 3DTV, remote robot control, remote medical service and so on. Stereoscopic video coding and transmission are the important parts of the research of stereoscopic video application. The stereoscopic video coding area requires the development of more efficient compression techniques to reduce the redundancy of a stereo image pair or stereoscopic video. H.264 is an advanced video compression standard, absorbing the advantages of the previous standards. H.264 represents a number of advances in standard video coding technology, in terms of both coding efficiency enhancement and flexibility for effective use over a broad variety of network types and application domains. The study subject of this thesis is the research on H.264 based stereoscopic video coding.In a single video system the compression is based on the intra-frame spatial redundancy and inter-frame temporal redundancy. Two pictures of the same scene taken from two nearby points form a stereo pair contain sufficient information for rendering the captured scene depth. That is, there is prodigious redundancy in the stereoscopic video inter-sequence. Therefore, in a stereoscopic system a more efficient coding scheme may be developed if the inter-sequence redundancy is also exploited. A typical stereoscopic video compression scenario includes the effective prediction of the right sequence frames based on both motion and disparity estimation. In general, there are two methods of implementing either motion or disparity estimation in stereoscopic compression applications. The first method, based on intensity processing, handles this estimation by the block matching algorithm (BMA).That is block-based stereoscopic video coding. The second method for either motion or disparity estimation, based on object segmentation, firstly defines or derives the features of the participating objects in the processed frame and then estimates the temporal or disparity field between corresponding frames. That is object-based stereoscopic video coding. Considering the characteristic of H.264, this thesis proposes block-based stereoscopic video coding scheme based on H.264, It includes five parts as follows:Firstly, this thesis presents a kind of stereoscopic video coding scheme based on H.264, which contains components of prediction (motion and disparity prediction), transform and entropy coding. The left channel is main view, and the right channel is auxiliary view. The left channel of the sequences is encoded as a H.264 bit-stream. The right channel is encoded by joint motion and disparity compensation based on block. Find the best matching block by full search under a matching criterion, which is the mean absolute error (MAE). Work out the corresponding residual block and motion or disparity vector. Compare residual block which is calculated by motion estimation with residual block which is calculated by disparity estimation to find the minor residual block and store the corresponding motion or disparity vector. The residual block and vectors are transformed by integer transform, quantized and coded by context-based adaptive variable length coding (CAVLC). It implements stereoscopic video compression.Secondly, the algorithm of H.264 based stereoscopic video coding is implemented on the H.264 official model JM9.0. Carry out comparison on three stereoscopic video coding schemes by the experiments. The experimental results show that the joint motion and disparity compensation is the most effective in compression performance. Do experiments on arrays whose image complicated degree are different. The coding scheme can achieve fine reconstructed image quality. It proves the stereoscopic video coding algorithm is effective.Thirdly, the optimization of disparity matching algorithm which affects algorithm speed is studied. Fast and efficient disparity matching of stereo video coding is a key technology in stereo video processing. On the basis of analyzing the correlation between disparity vector and motion vector, a fast disparity matching estimation algorithm for stereo video coding based on H.264 was presented. The experimental results show that compared with the full search algorithm, the fast algorithm can greatly reduce disparity estimation searching times and improve algorithm speed while the reconstructed image quality is close to the one by means of full search method. It optimizes the algorithm successfully.Fourthly, inter prediction technology of H.264 was analyzed in this thesis, and the optimization of inter prediction by adding B frames which affects reconstructed image quality is implemented. B frame can adopt in the past frame and the future frame at the same time being the predict reference frame. So it's called double direction prediction. B frame uses forward and backward motion compensation simultaneity. Using double direction prediction can achieve better prediction performance. The experimental results show that after adding B frames and employing IBBP encode method, the SNR of the right channel reconstructed image improves more than the one by means of IPPP encode method. It optimizes the algorithm successfully.Finally, Comparisons and analysis are made between the proposed method in the thesis and other stereoscopic video coding methods.(1) Comparing with the coding method for stereoscopic video with 3D-SPIHT The proposed H.264 based stereoscopic video coding method employs advanced motion compensation technology, such as tree structured motion and disparity compensation which partitions macroblocks into sub-blocks of varying size, seven different kinds inter prediction modes, quarter-pel accuracy prediction, multi-reference frame mode, weight prediction mode. These make motion and disparity compensation technology almost perfect. So its compensation performance is better than the one using 3D-SPIHT. The experimental results show that under the same bit-rate, the reconstructed image quality achieved by the proposed method is better than the one using 3D-SPIHT.(2) Comparing with the stereoscopic video coding method based on hierarchical block matching disparity estimation. The proposed method is implemented on the high profile of H.264 FRExt. This profile mainly aims at the special applies which have especially request for high resolution and high definition. So it can achieve much better compression performance. The experimental results show that the PSNR and the compression ratio (CR) of the stereoscopic video coding method based on H.264 is obvious better than the stereoscopic video coding method based on hierarchical block matching disparity estimation.(3) Comparing with the stereoscopic video coding method based on discrete wavelet transform (DWT)The proposed method employs the 4x4 integer transform. The transform unit is the block whose partition size is 4x4 but not 8x8 as usual. Because of choosing a smaller partition block size used in the transform, the division for motion object becomes more exact and the join residual in edges of the motion objects is decreased extremely. The experimental results show that under the same bit-rate, the PSNR of the stereoscopic video coding method based on H.264 is better than the stereoscopic video coding method based on discrete wavelet transform (DWT).As a conclusion, the proposed method in this thesis considers a block as a unit to perform prediction, transform, quantization and entropy coding to get rid of various correlations in the stereoscopic video. Also, it does disparity compensation to eliminate the stereoscopic video inter-sequence redundancy. The experimental results prove that the proposed method has better compression performance for stereoscopic video coding based on H.264.There are some points which need to improve further in the thesis such as how to enhance the compression efficiency under high PSNR for stereoscopic video with high-speed moving and how to implement object-based stereoscopic video coding.