Research On Theory And Application Of Electronic Image Stabilization

Video stabilization in dynamic sequence is an important research area of video processing and computer vision. It can improve the stability and fidelity of video sequence recorded from cameras on moving carriers, and has been widely used in military affairs and civil cameras. Electronic image stabilization (EIS) is the new trend of the video stabilization, which is the technique to determine the motion between frames and compensate it with image processing methods. In this dissertation, the author mainly focuses on the research of EIS on three aspects: motion estimation,motion decision and motion compensation. In especial, three image stabilization algorithms are proposed to deal with different motion types of camera systems. At the same time, the research results of global motion estimation is applied to moving objects detection in dynamic sequences, which is realized with background matching and frame difference. The main research work in the dissertation is as follows:1. The theories and key techniques in image stabilization are detailed. The basic geometrical principles of the camera and the transform models of image motion are introduced. Thereafter, the reasons and types of inter-frame motions are analyzed and a fast method to determine the camera motion type is proposed. Then, the motion estimation and compensation methods are detailed and analyzed. Lastly, the difficulties of EIS are discussed and the improving thoughts are summarized.2. Aiming to deal with the small dithering at the fixed scene, a hierarchical bit-plane matching (HBPM) is presented to realize fast image stabilization. The speed is highly improved due to the simple XOR operations instead of the SAD of block matching. It makes full use of multi-resolution pyramid matching with different bit-planes to find the translation vector. According to the high correlation of adjacent blocks'motion vectors and father-children of hierarchical blocks, the initial search point is adaptively chosen. The threshold is also selected to detect the motion accuracy to finish the search in advance. With all the local motions as a set, the motion with the largest number is detected as the global motion. Lastly, the dithering motion is computed as the accumulation of the inter-frame translations and compensated frame by frame. Experimental results show that this algorithm excels the block matching in speed while the error is below 0.5pixel and the result sequence is smooth.3. Aiming to deal with large translation dithering, a fast projection algorithm (FPA) is proposed based on gray projection of divided blocks. Firstly, the histogram equalization is applied in the original images to enhance the contrast. Then, the image is divided into separate blocks and projected in rows and columns, respectively. In the process of projection correlation, the three points'adaptive search method is presented to conduct the search towards the right direction to find the single minimum peak. Then, the median filter is applied on all the block motions to find the correct global motion with the largest number. Lastly, the inter-frame motions are filtered with the adaptive average filter and the error control is applied to select the original frame to be compensated. Experimental results show that this algorithm excels the traditional projection algorithm in speed while maintains the same accuracy. Even when the translation is as large as the 1/3 of the image size, it also accomplishes the real-time applications and high veracity.4. Aiming to deal with complex dithering, a global points'iteration (GPI) algorithm is proposed based on point matching. It is able to deal with translation, rotation and zooming and it's robust to local moving objects in the scene. Firstly, the points are selected in the reference frame and then each feature window is matched to find the corresponding points in the current frame. Then, considering the local moving objects and the covered or the disappearing points, the distance criteria is applied to eliminate the mismatched and the local points. Thirdly, the candidate points are brought into the motion model to make iteration with the least-square method. As a result, the points with large error are deleted gradually and the global points are sustained. Lastly, the Kalman filter is applied to smooth the motion vectors to obtain the dithering vector and the image mosaic is used to compensate each current frame. Experimental results show that this algorithm accomplishes high precision and speed. The error with the true motion is below half the pixel and the result sequence is smooth with high integrity.5. The frame difference can not preserve the foreground moving objects and the simple threshold can not realize adaptive segmentation of moving objects. Aiming to solve the problem, the variant blocks difference based on compensation (VCBD) is proposed to detect moving objects. Firstly, the global motion is estimated between adjacent frames to compensate the current frame. Thereafter, the difference based on variant blocks instead of pixels is made from-coarse-to-fine. Large size is used to divide the images and the median absolute difference is compared with the threshold to make fine divisions till the smallest size. At last, the blocks with large difference are labelled as the foreground moving objects. Experimental results show this algorithm is able to detect the moving objects from dithering video sequence.