Research On Video Stabilization Algorithm Based On Complex Motion

With the advancement of technology and the development of embedded devices,the types of portable shooting equipment have become diversified and the price has become cheaper.However,due to the incapability of the shooting staff,the lack of video stabilization equipment,and the influence of bad weather,the footage appears jittery.This jitter not only degrades the viewing experience,but also adversely affects subsequent processing analysis based on these videos.Therefore,video stabilization has great application value,which also is the current research hotspot in the field of video enhancement.At present,the methods of video stabilization for simple motion and conventional scenes have achieved significant achievements,but existing algorithms still face stabilization performance degradation or failure when dealing with complex motion videos.These complex motions include not only feature point mismatches caused by large foreground occlusions and diverse moving objects,but also motion estimation accuracy degradation caused by changing shooting environments and low-quality video.This paper conducts in-depth research and analysis on the above-mentioned complex motion video stabilization problems,and proposes targeted algorithms respectively.To sum up,the main work and research results of this paper are as follows:1)A video stabilization algorithm combining superpixels and improved K-means clustering is proposed.This algorithm mainly solves the stabilization problem of videos containing multi-object motion and large foreground occlusion.When the traditional method based on features processes the above-mentioned complex motion videos,the features will be mismatched due to the disturbance of local motion,which will affect the accuracy of global motion estimation and lead to a decrease in the stabilization performance.Unlike most existing algorithms that only use Random Sample Consensus(RANSAC)to eliminate mismatched point pairs,this paper proposes a coarse-to-fine local motion eliminate method.The method uses adaptive superpixels to naturally separate multiple foreground objects or different regions of the object from multiple background regions with different depths.According to the difference between the foreground and background motion vectors between frames,the superpixel block motion space and the feature point motion space with certain information correlation between image frames are constructed.And the K-means clustering is used to separate local motion and global motion.The method in this paper has been simulated and verified on different types of videos.Compared with the other stabilized method based on feature points and single matrix in the past 6 years,it has better performance.And the average structural similarity and average peak noise are 0.24 d B and 2.31 d B higher than original video,respectively.2)An unsupervised staged video stabilization algorithm is proposed.When dealing with changing shooting environments and low-quality videos,the current traditional algorithms will face the problem of decreased or invalid motion estimation accuracy,resulting in the degradation or failure of stabilization performance.To solve this problem,this paper proposes an overlapping content-aware homography estimation network.The network reduces the influence of local motion on global motion matrix estimation through the proposed overlapping content-aware module and overlapping feature loss term.And by exploiting the powerful learning representation ability of convolutional networks,we solve the problem of low-quality video motion estimation accuracy degradation,and achieve sub-pixel accuracy in public datasets.In addition,most methods based on constrained smoothing use fixed weights to suppress high-frequency noise.However,due to the complexity of the shooting environment and motion,more iterations are required to achieve the same smoothing effect,thereby increasing the stabilization time.To solve this problem,this paper proposes a dynamic weight generation network,which can adaptively generate weights according to the perceived trajectory information,and can achieve the performance of traditional constrained optimization with fewer iterations when dealing with complex motions.Through staged training and comprehensive integrated testing,a stabilization network model based on a learning mechanism is formed.On the video stabilization datasets,the algorithm in this paper achieves results close to the traditional method,and in the self-built difficult scene datasets,it achieves more robust results than the traditional method.In addition,compared with other deep learning-based methods,the algorithm in this paper has a faster processing speed on the NVIDIA RTX 2060 SUPER graphics card,which can reach 40.4FPS.To sum up,this paper conducts in-depth research on complex motion video stabilization problem,and proposes a video stabilization algorithm combining superpixels and improved K-means clustering,and an unsupervised staged video stabilization algorithm.The stabilized effect was verified on the self-built datasets and public datasets.A tracking experiment is also performed on the image stabilization video,which further proves the significance of stabilization.