A Study Of Electronic Image Stabilization Algorithms And Visual Tracking Algorithms

As camera systems are widely used in the electro-optical reconnaissance system,hand-held mobile device and monitoring system etc., the clarity and stabilityrequirements for the captured image are getting more and more important. Theinstability of camera equipment, however, will cause the incoherent content of thecaptured image sequence, which becomes blurring and shaky. This image sequence notonly has a poor visual effects, making the monitors fatigue, but also affects theirjudgments on the events observed. Moreover, this jitter image sequence will make theimage post-processing such as visual tracking and video compression, which use theinformation from the image, very difficulty. Therefore the stabilization of camerasystems is a very important issue. Electronic image stabilization is one of theapproaches that remove shaky effect from the captured video to improve its visualquality. This dissertation is mainly focused on the development of image stabilizationalgorithms for videos with moving objects. And the real-time image stabilizationalgorithms with complexity motion model are explored. Since visual tracking has beenwidely applied to visual surveillance and human computer interaction, the visualtracking is also investigated in the final part of this dissertation. The major contributionsof this dissertation are outlined as follows:1. In digital image stabilization, the global motion is estimated by exploiting theimage information without using motion sensors. Therefore the quick and precise globalmotion estimation is the core of digital image stabilization. The disturbance of themoving object, however, usually results in incorrect global motion parameters. To tacklethis problem, an image stabilization algorithm based on the inverse compositionalalgorithm and a detection blocks dynamical selection method is proposed. The inversecompositional algorithm which is based on a global optimal rule is less sensitive to localeffects. To remove the disturbance of large moving objects, a motion detection methodbased on background compensation and frame differencing is used to dynamicallyextract the detection blocks from background, with which the global motion parametersare retrieved. The experimental results show that the average Inter-frameTransformation Fidelity of the stabilized videos obtained with the proposed algorithm ismuch higher than that of the original videos. And the proposed algorithm has a goodreal-time performance.2. The robustness of the video stabilization algorithms based on the traditionalmodel parameters fitting is usually low. In view of this, a Harris corner feature-based image stabilization algorithm is proposed for videos with moving objects by combininga background feature block matching with a histogram clustering method．The obtainedglobal motion parameters are exploited for compensating for the background motion.And the frame differencing method is used to divide the reference frame intoforeground and background blocks. The reference feature block on the background ismatched with the feature blocks of the current frame to roughly remove the featurevectors on the moving object. By using a one-block-to-multiple-block matching strategy,the reference feature block is matched with the feature blocks of the current frame in thesearch window centered on the reference block to prevent the local optimal problem,and a sparse motion vector field is built. Then, the un-removed foreground vectors anderroneous vectors in this vector field are filtered out using a histogram clusteringmethod. The left inliers are used as the input of the Least-squares algorithm, and thefinal global motion parameters are obtained. The experimental results indicate that theaccuracy of the proposed approach is as high as those of the state-of-the-art algorithmsand techniques. Moreover, it has a higher robustness to moving objects.3. The real-time performance of the current image stabilization algorithms with acomplexity motion model is usually low. In view of this, a complexity motionmodel-based image stabilization algorithm is proposed by using the local binary featureORB. The ORB feature consists of the oriented FAST detector and the rotated BRIEFdescriptor. The feature points from two consecutive frames are firstly extracted with theoriented FAST detector. And the feature points are described with the rotated BRIEFdescriptor. The keypoint matching is performed using hamming distance betweendescriptors’ vectors and a distance ratio, namely ratio of closest neighbor distance to thesecond-closest neighbor distance, can be checked against a threshold to discard falsematches, which is similar to the Euclidean distance matching in the SIFT. The globaltransformation parameters between the consecutive frames are refined by using theProgressive Sample Consensus (PROSAC). The experimental results indicate that theproposed image stabilization algorithm is very robust. And its real-time performance ismuch better than that of the SURF. The real-time image stabilization has been realizedfor some image sequences.4. The real-time performance of the particle filtering tracking algorithms based onhistograms is usually low. In view of this, based on the integral histogram technique,two low complexity particle filtering tracking algorithms are proposed for the pedestriantracking and head tracking, respectively. An integral orientation histogram construction method based on the sparse property of the image gradients is used for the extraction oforientation histograms, with which an orientation histogram matching-based responsemap is built. The observation information from this response map is extracted for theconstruction of proposal distribution function (importance function). Moreover, theproposed particle filtering tracking exploits the layered particle sample strategy, suchthat the state dimension can be reduced and the state space exploration can bespeeded-up. Simulation results show that the proposed tracking method has lowercomputational complexity than that using straightforward histogram extraction methodwhen using large number of particles for tracking large objects. And the proposedtracking method has a high robustness to the illumination variations, scale variations,head rotation and quick movement.