Research On Video Post-Processing

With the development of the society and advancement of technology, the demand of all kinds'information is growing rapidly. The multimedia content which contains different formats, such as video, still image, audio, speech and text are becoming the main medium of information. Among all the formats, video and still image take the dominant role. The module of video post-processing, which acts as an interface between the decoder and display device, is responsible for converting the video format and performing image processing. It plays an important role in improving the image quality and enhancing the viewing experience of the final receiver.Frame rate up-conversion is a kind of technique that is used to convert the video frame rate from a lower number into a higher one. It can be used in the decoder side to recover the video frames skipped in the encoder for meeting the low bit-rate bandwidth requirement. The converted video is much fluent and provides better visual quality. Error concealment techniques are always implemented at the decoder side in video applications. It can estimate and recover the lost information of the video which is caused by transmission over erroneous channel. The decoded video quality is greatly improved. Meanwhile, three dimension television (3D-TV) as a newly developed medium which can provide an enhancement of the viewing experience is becoming the new research area of video post-processing.In this dissertation, the main theory and the international standards for video coding is briefly reviewed first. After that, the decoder and post-processor co-architecture based on H.264 standard is proposed, in which the decoder feeds coding information to the post-processor. Based on the coding information, the features of the local image are estimated and can be used to improve the performance of the post-processor. The main research topics of this dissertation are some algorithms under this co-architecture and some new applications in the 3D video system. It can be summarized as follows:A novel frame rate up-conversion based on spatial and temporal correlation integrated, by using the motion vector from the decoder is proposed. The coding-type information, the spatial correlation in the motion vector field and the temporal correlation in the before and after frames,the spatial correlation in the interpolated frame, and the constant motion velocity model are used together to estimate the motion vectors of the interpolated frame. The algorithm takes the similarity of the related blocks in the before and after frames and the spatial correlation in the interpolated frames into consideration, which greatly decreases the blocking artifacts and ghost effects, and keeps the edge of the object clearly.A multi-model based temporal error concealment algorithm is proposed. Improved temporal correlation model and constant motion velocity model are adaptively combined according to the coding information of the neighboring area. Then it is used to interpret the motion of the lost area for recovering the motion vector. After that, the lost pixels are concealed by using multiple-hypothesis motion compensation. The temporal correlation of the neighboring area around the lost block and the coherence of the motion in adjacent frames are considered together. It helps to reduce the error propagation and improve the quality of the recovered video.A motion vector correction based frame recovery algorithm is proposed. The spatial correlation among motion vectors and the pixels'information in the reference frames are used to correct the motion vectors in the reference frame. After that they are used to estimates the motion vector field of the lost frame. Finally, adaptive overlapped block motion compensation is used to reconstruct the lost frame.An inpainting based multi-view image generation algorithm for 3D video system is proposed. Different from conventional algorithms which focus on the dealing with the depth image, the inpainting is proposed to fill the blank area after the projection.A depth image based temporal error concealment algorithm for 3D video system is proposed. By considering the depth information of the lost marcoblock, the inner structure of the lost marcoblock is estimated. The foreground area and background area will be separated and concealed respectively.In order to evaluate the performances of those proposed algorithms, many experiments are done with all sorts of sequences. Experimental results show that the proposed algorithms outperform other techniques in both objective and subjective visual quality.