Research On Frame Rate Up-Conversion Algorithm

Frame rate up-conversion(FRUC) is a video post-processing technique that can increase the frame rate of source video into a higher frame rate by inserting several intermediate frames. In high-definition digital television systems, FRUC technique can remove judder and motion blur of low frame rate videos, which provides more natural viewing experience. While in the bandwidth limitted video applications, this technique can be used to recover the frames that are skipped by the encoder, which can effectively eliminate the sense of scene jumping at the receiving end. With the increasement of people’s demand on the visual quality, and the frequent video interaction between various multimedia services, FRUC technique has became one of the hot spots of the video post-processing research field. The accuracy of the motion vector(MV), the quality of the motion compensated interpolation, and occlusion and ambiguity region handling are the focus and difficulties of FRUC.As an important video post-processing technique, FRUC has a wide application prospect for both conventional 2D video and emerging 3D video applications. This thesis takes frame rate up-conversion of 2D video as the main research content, and uses the block-based motion-compensated frame interpolation(MCFI) technique as technical route. By deep analysis of the difficult problems such as true motion estimation, motion vector field(MVF) post-processing, motion compensated interpolation, and occlusion handling, this thesis proposed some effective solutions. In addition, different from 2D video, 3D video has extra depth information. This thesis makes preliminary study on the technique of using depth information to improve the frame interpolation quality of the video plus depth(V+D) format 3D video. To sum up, the main work and innovations of this thesis are as follows:As to the unidirection block motion estimation problem,based on the Bayesian theory, an adaptively prior constrained unidirection block motion estimation algorithm is proposed. By considering image local features, the proposed algorithm adjusts the smooth strength adaptively to avoid the motion field over-smoothing problem of conventional methods.As to the motion vector field post-processing problem, by introducing interpolation artifact information as new reliability metric, this thesis proposed a new post-processing method. Compared with existing methods, the proposed algorithm is more efficient in handling the unreliable MVs which are in the texture less area or appearing in clusters.As to the block-based MVs are lack of precision along object boundaries, according to the kernel regression theory, a joint trilateral filter is proposed to transform the block-based unidirection MVF into pixel-based bi-directional MVF, which effectively improves the precision of the MVF along the object boundaries. Then, by utilizing the pixel-based bi-directional MVs, this thesis further present a support window based bi-directional motion compensation algorithm to suppress the artifacts within the interpolation result.As to the problem that occlusion and ambiguity regions cannot get accurate bi-directional MVs, by utilizing temporal motion consistency metric, this thesis proposed a multi-frame based occlusion and ambiguity region handling method. This method can detect the occlusion and ambiguity regions within the intermediate frame plane effectively, and the bi-directional motion selection and compensation algorithms can be further improved with the help of the detection results.By analysizing 3D video characteristics, based on 2D video motion compensated interpolation technique, this thesis proposed an FRUC method for V+D video. By fully utilizing depth information, the porposed method solves edge detection and foreground and background determination problems effectively. With the improved MV processing and motion compensated interpolation algorithms, the integrity of object structures and the accuracy of the depth information within the interpolation results are all well kept.Various comparison experiments are implemented on the the performances of those proposed algorithms. Experimental results show that the proposed algorithms outperform other benchmark methods both subjectively and objectively.