Real-time Video Mosaic Based On Pan-tilt Controlling

With the application of video processing becoming more and more common and important, such as virtual reality, video surveillance, video compression and video editing, video panorama technique has improved greatly as being widely used in these domains. However, because of the complexity of video, video mosaic becomes a difficult problem. This paper presents the real-time implementation of video panorama based on pan-tilt controlling. The process of establishing video panorama encompasses three steps, which are global motion estimation, motion object segmentation and image mosaic. In this paper, we study the technical problems of each step in detail and propose improved methods.To construct a video panorama, the first step is to estimate the global motion parameters. Chapter Two analyzes the problem in detail. This paper studies and implements phase correlation and SIFT method. After analyzing and comparing the two methods, Chapter Three proposes an effective method of image mosaic. First, phase correlation is used for pan-tilt calibration, which can roughly compute the overlapped region between two frames. Second, it extracts SIFT features in the overlapped region from each frame and matches those features between frames with RANSAC, which uses voting to ensure the robustness and correctness of corresponding results. The approach can compute the affine matrix of video frames rapidly and accurately, realize the real-time performance of the system.Moving objects in the video sequence interfere in the result of video panorama. Chapter Four studies the motion object segmentation when moving objects have overlap regions between two frames. First, locate the rough regions (called regions of interest in the paper) of moving objects by frame difference method. And then segment the moving objects in those regions of interest based on Canny arithmetic operators in one frame.Chapter Five describes the whole implementation of video panorama mosaic, presents a new technique for the creation of a sequence of mosaic images from an original video shot. With this technique, each mosaic image M_i is iterativelyupdated using the previously computed mosaic image M_i-1 and the current image I_i,and it can effectively minish accumulated error and image blur. Further more, Chapter Five presents a new method for handling moving objects, removing moving objects inM_i-1 whereas keeping moving objects in I_i, which helps gain a panorama withmoving objects in current image.The final Chapter summarizes the whole paper, draws conclusions and proposes several potential future directions based on the current work.