Research On Real-time Stitching In Aerial Video

Video Stitching is a digital multimedia technology, has a broad application prospects in the intelligent monitoring, defense, transportation and other fields. Using the Image preprocessing, image registration and image fusion technology, videos taken by different perspectives video cameras while having overlapping relationship, eventually, were joint together into a video with a wide perspective. The core technology of Video Stitching is image registration, traditional matching algorithms exist the disadvantage of large amount of calculation and poor real-time performance. Jitter between video frames greatly influenced the visibility of the video out. The following research works are carried out:Firstly, SURF algorithm generates a 64-dimensional vector descriptor via calculating Haar wavelets response on the area near the feature point. Shortcoming of computationally intensive exists and the high dimensional vector descriptors affect the matching speed of feature points. In order to solve this problem, this paper used the FREAK features description algorithm. FREAK algorithm generates a 128-dimensional binary descriptor, extracting 128 pairs of binary intensity tests on Retina model, greatly reduces the amount of features calculation. Using Hamming distance to measure the similarity between binary descriptors, save more time when matching feature points. And for the shortcoming that the central symmetry model is too sensitive to image geometric distortion, the paper presents anisotropic feature description algorithm to correct the Retina model adaptively. Experimental results show that the accuracy rate of feature matching is more than 80% and the speed of matching algorithm improves 10 times than the classical algorithm-SURF, achieve real-time performance of the algorithm.Secondly, the paper describes the pyramid KLT algorithm and realizes the feature points tracking. Using a sub-pixel interpolation algorithm makes KLT tracking accuracy of sub-pixel level. No longer need to extract feature points from frame to frame, KLT tracking algorithm is faster than Feature point extraction algorithm.Thirdly, the paper describes the Random Sample Consensus (RANSAC) algorithm, obtains the optimal consistent set and transformation matrix using RANSAC, and calculates iteratively the optimal transformation matrix using LMA algorithm. We introduced the gain compensation and the multiband fusion algorithm, got a wide perspective mosaic result with rich natural details.Finally, we achieve aerial stitching system real-time system. We adopt Kalman filter to estimate the active scanning of the video cameras, realized the motion compensation of the video stitching system. Due to the overlapping area is too small, the error of transformation matrix increases. According to the relationship between the transformation matrixes, the paper expand the proportion of the overlap region by matching the front and rear frames instead of matching left and right frames, eliminated jitter between video frames. The experimental results show that this system has realized the real-time stitching of the aerial video with a visible video out.