Video Editing Based On Mosaic

The rapid advance of the information technology and electronics has introduced many devices can capture and record video, such as, Video camera or Web camera. These devices generate huge amount of video data, and raise requirements of video processing technologies. To manually process such amount of video data is impracticable, many researchers study on automatically video indexing and video editing technologies. Considering a video sequence is a set of connective image frames, to correctly manipulate the video sequence we should keep the correspondence between frames and thus to extract the correspondence between video frames at first. The image correspondence and the along with segmentation are two of the most hot and difficult problems in computer vision and image processing domain. We deal with the problems with a new appliance, to create video scene panorama. By exploiting the intrinsic structure of the video scene, we propose to construct panoramas of video sequences. With the panoramas, a user can easily and efficiently edit video sequence with manual and automatic approaches. The process of establishing video panorama encompasses three steps, which are motion parameter estimation, motion object segmentation and image mosaic. In this paper, we study the technical problems of each step in detail and propose algorithms of the generated panorama editing. Moreover, we describe several practical appliances of panorama based video editing.To construct a video panorama, the first step is to estimate the camera motion parameters and establish the project correspondence between following video frames and projections from 3d space to the frame planes. Chapter 2 describes the approach to solve the problem. We propose to extract Harris corner features from each video frames and match those features between frames with RANSAC. Our approach uses voting to ensure the robustness and correctness of corresponding results. And with the results, we can compute the projective matrix of video frames and the intrinsic and extra parameters of cameras.Moving objects in the video sequence interfere in the result of video panorama. The chapter 3 describes how to segment and remove the moving objects from video sequence. A two steps approach is proposed. Firstly, we compute Frame difference to estimate the initial positions of the moving objects. Secondly, we use the Mean-shift to segment the video frames and combine the segment results by refining the initial estimated moving object with the computation of a graph cut. During the calculation, the segmentation result of the previous frame is introduced as a restriction of the current frame segmentation. And thus it assures the continuous and smooth segment results. Our approach incorporates the advantages of the Frame difference and Mean-shift. It can quickly locate the moving object and precisely segment the moving objects from video frames.After motion estimation and motion segmentation, we can mosaic the video frames to derive the panorama. Chapter 4 introduces two mosaic models, planar projection model and cylindrical projection model. The cylindrical projection model assumes the camera did in-plane motion during the capturing process. But in real circumstance, the hand-held camera may often tilt and it raises the image curl during the mosaic process. For this, we propose an optimization algorithm to compute an optimal cylinder axis. The approach successfully eliminates the image curl. Furthermore, luminance of video frames may change and it introduces the difference of white balance and exposure parameters among video frames. Then the color spectrum of the video frames is not consistent. We proposed an efficient algorithm to rectify the color spectrum. Itextracts the corresponding points and uses the histogram of corresponding points to compute the color correspondence. Our approach exceeds traditional approaches in robustness because it can reduce the errors introduced by the wrong correspondences.With the constructed panorama we can edit the video sequence by editing the panorama image. The chapter 5 represents three image editing methods: Manually interactive image clone; Image in-painting; Image texture synthesis. We consider the first method could be used to fill the assigned region; the second approach could be used to edit and restore the smooth or narrow band region; the third approach could be applied on image areas with normal textures. We adapt Image in-painting algorithm with a real time implementation and define a novel distance measurement to enhance the texture synthesis algorithm with the reducing color dependency. The experiments demonstrate the efficiency of our algorithms.Chapter 6 gives three real appliance of video editing with video panorama. They are motion panorama, moving objects removing and video restoring and re-editing. Several problems existed in the real video editing procedure. The problems are, for example, video twitter and black edges introduced by motion compensation. We provide the approaches to tackle the problems and demonstrate the performance of the approaches with extensive experiments.The final Chapter, chapter 7 summarizes the whole paper, draws conclusions and proposes several potential future directions based on the current works..