Stereoscopic Video Stabilization Based On Digital Video Stabilization Technology

Advanced stereo shooting devices are developing from a professional level to a consumption level.Thus,there is an increasing need for stereoscopic video stabilization with the springing up of stereoscopic videos and movies.Directly watching stereoscopic videos with jitter will generate visual fatigue.Although there are many mature monocular video stabilization methods in this field,stereoscopic video stabilization has not been studied broadly.Simply applying traditional monocular video stabilization methods to stable left and right video independently will lead to awful viewing experience.On the one hand,horizontal parallax will become unsteady and vertical parallax will be introduced.On the other hand,frame synchronization might exist.Therefore,this paper aims to study the special aspects of parallax consistency and frame synchronization of stereoscopic video stabilization based on monocular video stabilization.Firstly,a stereoscopic video stabilization method with strong parallax constraint is presented.Then,with frame synchronization taken into account,an improved stereoscopic video stabilization method is further studied.Stereoscopic video is stabled with left-and-right consistent parallax constraint after each sequence is smoothed by monocular video stabilization method which is based on structure-from-motion algorithm.Monocular video stabilization smoothes the camera motion parameters obtained from structure-from-motion algorithm.There is a high probability that horizontal parallax in the time domain will become unstable and vertical parallax distortion will be introduced since independent monocular video stabilization damages the original left-and-right consistency.To solve this problem,horizontal and vertical parallax constraints are added.Horizontal parallax constraint includes similarity and stability part.The original horizontal parallax isn't quite stable due to camera shake.If the difference is made between the transformed and the original parallax directly,it is likely that the parallax jitter is reserved.Therefore similarity constraint is resented in the form of the difference between the parallax after similarity transformation and the average parallax of five original neighboring frames.Stability is represented by the changing amount of horizontal parallax of adjacent frames.As for vertical parallax,this paper makes it to be as close to zero as possible to provide a more pleasant viewing experience since human eyes are very sensitive to the vertical parallax.The monocular video stabilization result is combined with binocular disparity constraint.In the time domain,projection points of the original sequence are transformed to the points of the smoothed sequence by similarity transformation.In space,they satisfy the binocular disparity constraint.The optimal similarity transformation can be acquired by L-M method.And the stable stereoscopic sequence can be achieved after the original sequence is transformed.Usually multiple video stabilization is based on the precondition that they are frame-synchronized.But in fact,asynchronous problem might exist.Therefore,frame synchronization is implemented before the parallax constraint is added.The momentary rate of camera motion isn't uniform since the motion contains intentional motion and unintentional shakiness.Camera rotation parameters obtained from structure from motion are converted into three Euler angles.The changing rate of each angle per frame can be obtained through angle difference.Since the objects share the same motion in the two viewpoints,the Euler angles should share the same changing rate.But when the two sequences are asynchronous,a slight delay of the angle variation will lead to the asynchronous angle changing rate in the two views.A cost function is put forward based on this.And the most matched left and right frames are found when the value of the cost function is the smallest.Then parallax constraint is added after the two sequences are synchronous.We have tested several different shaky stereoscopic videos,and the results show that our method is effective.