Research On Stable Virtual Viewpoint Generation Of 3D Video Based On Optical Flow Method

With the vigorous development of 3D technology,3D video display becomes more and more popular in everyone's life.In comparison against wearing glasses 3D technology,naked eye display technology provides a better visual experience.Naked eye display solutions usually use existing binocular 3D video to draw a plurality of intermediate viewpoints,and then synthesize the final multi-view 3D video.However,in this process there always has some jitter because of factors such as unstable stereo matching.How to effectively generate intermediate virtual viewpoints and ensure the stability,this is a meaningful research.This paper analyzes the causes of unstable virtual viewpoints and use optical flow tracking to improve jitter suppression,proposed two stable virtual viewpoints generation solutions.The contents and results are as follows:1.Proposed a stable virtual viewpoints generation method based on sparse optical flow tracking.This method is composed of three parts:feature points extraction,intermediate viewpoints generation and feature points tracking.Adopt SIFT algorithm to extract and match the feature points of image frames,then use the Delaunay triangulation and Direct3D technologies quickly and efficiently generate intermediate virtual viewpoints.Use a combination of Gaussian pyramid and Lucas-Kanade sparse optical flow method to track the feature points.To ensure the stability and reliability of the feature points tracking process,a tracking period is introduced.Experiments show that this method is fast and generate high-quality virtual viewpoints.Especially,it has effectively improved the unstable situation of the generated intermediate virtual viewpoints.This solution is convenient in practical application for avoiding the depth map generation and the hole filling problem.2.Proposed a stable virtual viewpoints generation method based on dense optical flow tracking.This method is composed of three parts:optical flow tracking,intermediate viewpoints generation and the horizontal parallax calculation.Utilize the combination of image pyramid and TV-L1 optical flow tracking to obtain the horizontal parallax of stereoscopic image pair,thus presents a method to calculate the horizontal parallax of current frame which is according to the inter-frame motion information.Using the parallax offset method can get multiple intermediate viewpoints.Introduced a tracking period,in order to reduce the time-consuming,first apply down-sampling to the original images and then interpolate the generated intermediate virtual viewpoints back to the original image size.Experiments show that this method can generate good intermediate virtual viewpoints,make up part of the fuzzy phenomenon that in the sparse optical flow method and also guarantee the stability of the video frames.The two proposed methods can effectively generate stable and continuous virtual viewpoints.Both methods have their own strengths and complement each other in a certain scene,it has some practical significance.