Research On Multiple Levels Of Video Abstraction

Research on abstraction of image and video is one of the hot spots on Non-Photorealistic Rendering (NPR) recently. It has highly important application in many fields, such as remote education, entertainment, machine learning and recognition. Different from realistic image and video, image and video abstraction studies how to express certain visual information more efficiently as the designers are willing to, while discarding redundant details, emphasizing meanful parts according to user's intention. Besides, abstracted image and video has much advantage in storage and transfer speed.In the field of image abstraction, predecessors have achieved attention based multi-level rendering of photograph using eye-tracking device. Accroding to the importance of local information to observers, they generated different level of abstraction in different position. They represented specially concerned regions by abundant colors and obvious silhouettes, and rendered other parts sketchily, only preserving the rough feature. In the respect of video abstraction, research work in existence mainly did research on automatically producing stylized animation from video. There are several video based apporoaches, including optical flow field, target recognition and energy minimization, and 3D model based stylization rendering framework. However, majority of these apporoaches only discuss how to maintain frame-to-frame coherece of the generated animation, but not incarnate the thought of multiple levels of abstraction in a single picture. In many application fields, such as remote education, entertainment and machine learning, image and video abstraction in different levels accrording to the importance of the content is indeed required. It reflects the concern of users to the objects.In this paper, we mainly study the multi-level abstraction of image and video. First of all, we extract the feature of image and video at multiple scales, building an image pyramid, and then we segment each level of the pyramid into numbers of regions, which are filled with constant colors severally. We construct a hierarchical region structure, as an intermediate representation for multi-level rendering. After that, according to users' attention field, we add finer strokes on to concisely rendered canvas. This attains the goal of multiple levels of abstraction. Based on current abstraction technique of still image, we have done some research respectively on temporally coherent rendering method of dynamic image abstraction and video stylization.The content and contribution of our research can be summerized into the following two phases:1. We build a dynamic interactive abstraction framework that is available to browse an image when users' attention field is moving. Compared to existing methods, this paper begins from human visual perception characteristic, and considers the eye-tracking interaction with observers. While the attention focus is changing, the abstraction level changes consequently if necessary. We generate a series of successive frames, controlling the coherence by kinds of means.2. We build attention-based video abstraction rendering approach. The method of the paper is that, using anisotropic kernel Mean Shift segmentation algorithm, we process the entire three-dimension video data by different level of abstraction. Also we work with visual perception, and research the relationship between salient importance and level of abstraction. We transform a series of video sequence into cartoon-like style. The temporal coherence is maintained, while the level of abstraction is adjusted with local importance of the content to observers.At last, this paper demonstrates that the approaches above can obviously enhance both temporal and spatial coherence, and improve the visual impression of images and video. Based on these work, we can further study the multi-level video abstraction with other kinds of stylization and the processing of more complicated video in future-video with motion blur, fade in or out, etc.