Research On Video Stabilization For Traffic Videos Containing Complicated Moving Scenes

As the information technology develops very fast and smart devices become more and more ubiquitous,videos are being used to record people's daily lives.But most of video makers are just amateurs and lack professional devices and photography techniques.Thus the videos produced by non-professional makers often suffer from annoying jitters due to the shaky motion of unsteady cameras.Particularly,jitters of traffic videos captured by vehicle-mounted cameras are more violent and make the post-processing tasks,such as vehicle license plate recognition and target tracking,very difficult.Although video stabilization methods have made great progresses,some challenging problems still remains when processing traffic videos containing complicated scenes.The main focus of this thesis is to stabilize traffic videos containing complicated moving scenes.The shakiness of videos originates from jitters of the motions of cameras.So shakiness could be filtered out when camera motions can be estimated correctly.However,the accuracy of jitter estimation may degrade severely in the case complicated scenes.Actually,it is nontrivial and very challenging to correctly estimate the camera motions for rapidly moving scenes containing large moving object,multiple moving objects and parallax resulting from significant depth variation.For these challenges,especially the degradation of precision and increase of processing time,this thesis proposes some specific algorithms.Its main work and contributions can be summarized as follows.1)A feedback based video stabilization method for traffic videos is proposed.Although there are some solutions for large moving objects or parallax among the existing video stabilization literatures,the complicated scenes containing both large moving objects and parallax cannot be well handled.In this thesis,parallax and motion of moving objects are modelled and analyzed mathematically.Then a feedback based foreground trajectory judgment method is proposed,where the trajectories corresponding to moving objects and parallax are recognized by reprojection errors of homography matrices of the feature trajectories.In order to improve the accuracy of foreground trajectory judgment when large moving object appears,feedback mechanism is introduced so that we can filter out the known foreground trajectories from potential background trajectories.All of the background trajectories are low-pass filtered to remove the shakiness and obtain the smoothed background trajectories.Then a homography matrix is computed from the original and the smoothed background trajectories,which is used to obtain the stabilized version of the input videos.Since the foreground trajectories corresponding to foreground objects and parallax have been filtered out from the background trajectories,enough stabilization can still be provided when large moving objects and parallax exist simultaneously.2)Three methods are proposed to enhance the robustness of our traffic video stabilization method through increasing the number of background trajectories.As we know,the feedback based video stabilization is based on background trajectories.Hence the precision of video stabilization greatly depends on the number of background trajectories.Nevertheless the number of background trajectories might severely decrease for many reasons,such as misjudgment due to strict decision rules,large moving object or multiple foreground objects which may lead to serious occlusion to background areas,the rapid movement of cameras.The performance of video stabilization can seriously degrade when the number of background trajectory decrease severely.Thus we propose three refinement methods to increase the number of background trajectory,including background trajectory recovery,block based adaptive feature detection algorithm and adaptive ad.justment of the number of feature point.These methods can specially address the three situations above so as to greatly increase number of background trajectory and decrease the number of foreground trajectory,which can remarkably improve the robustness of our feedback based video stabilization method to traffic videos containing complicated scenes.3)A derivative-based foreground trajectory judgment algorithm is proposed.Multiple homography matrices must be computed when feedback based foreground trajectory judgment is performed.The computation of these homograph matrices are time-consuming because it requires many SVD decomposition operations.In order to speed up our video stabilization method,we propose a derivative-based foreground trajectory judgment algorithm.Under the new algorithm,a foreground trajectory is recognized base on the reprojection errors of linear model of derivatives of feature trajectory rather than the homography matrices.The feedback mechanism is also introduced to improve the accuracy of foreground trajectory judgment when large moving objects get close to the camera.In order to verify the performance and advantages of our methods,all of the methods have been tested through many traffic videos.The experimental results show that our feedback based video stabilization method performs better than the existing methods,while the three refinement methods dramatically increase the number of background trajectories and decrease the number of foreground trajectory.Meanwhile,the derivative-based foreground trajectory judgment method significantly decreases the processing time of foreground trajectory judgment with only minor performance degradation.