Content-Based Video Panoramas

With the development of multimedia technology, digital video has become an important way to send messages and entertain in our daily life. Video sequence contains a wealth of information. It can quickly and intuitively express essence of video. But we always need to use a large area to store the video data, because it has a large amount of overlapping information between image sequences, which brings us many great difficulties of data storage, retrieval and network transmission. How to eliminate redundant information between image sequences, and use a simple image involved the main content of the expression is our target. Panorama is a useful form of video abstraction for obtaining a high resolution picture which owns a wide field of view. To some extent, it also can retain the video time information. Video panorama can be widely used in video compression, video surveillance and video texture generation.Panoramic technique is mainly divided two categories:image-based registration method and manifold-based strip stitching method. Image-based registration method is more suitable for the smaller number of pictures which has a smaller overlap between pictures. Manifold-based method is applicable to video which has large overlapping areas and a large number of image sequences. However, traditional manifold-based panoramic techniques can’t retain certain key areas of content in the panorama and it is hard to present the whole video motion information. The worst is too slow to achieve. Therefore, we promote a reseach of content-based panoramic technology. The main work and results of this paper are as follows:(1) For the video with partial motion, it is hard to predict where the moving object is and what the specific state of object is in panorama based on traditional manifold-based panoramic techniques. It is difficult to meet the needs of users want to reserve a specific state of moving object in panorama.In this paper, we propose a content-based video mosaicing method on the basis of the original. Combined with the salience detection, our method achieves the target of keep a particular object in panorama at a time with edge-constraint shortest path algorithm.(2) As the single object-constraint panorama only retain one state of the moving object a certain time, which leads to loss other motion states in the video sequence, this paper proposes a new motion panoramic technique. Combined with the key frame extraction technology, we generate many panoramas with content-based panoramic technique based on the reserved areas respectively. In order to merge the foreground objects with different states and the panorama without constraint together, KNN algorithm is used to extract moving objects in the foreground with in an interactive way. This paper also sets increments (decreasing) intensity of the object to express the movement information more clearly.(3) For video data always has a large number of redundant information, this article proposes optimization methods from both serial and parallel mosacing technique. Serial accelerated method combined time-space Gaussian pyramid structure for down-sampling, bottom-up hierarchical solve the shortest path. Parallel accelerated method mainly based on CUDA parallel computing, improve the speed of mosaicing by parallel implementation of the adjacency matrix construction and shortest path search algorithm. We analysis these two optimization methods and compare the pros and cons between them.