Research On Dynamic Texture Modeling Algorithms For Recognition And Synthesis Tasks

Dynamic textures(DTs)refer to sequences of images of moving scenes that ex-hibit certain stationarity properties in time.The time-stationarity of DT demonstrates regularity and repetitiveness in statistics,which is a generalization of static texture sta-tistical characteristics extending from image space to time domain.Dynamic textures are widespread in natural scene,such as smoke,waves,traffic,and so on.Dynamic tex-tures present a visual dynamic process that is a fundamental part of natural scenes and commonly found in video data.The underlying temporal variation process in dynamic textures can deliver richer visual information,play an important role in human visual perception,and offer crucial cues for solving a wide range of computer vision issues.Dynamic texture modeling is to construct the mathematical models and descrip-tive algorithms of dynamic texture representation for recognition,synthesis and other related visual tasks.Different from the lack of motion clues in static images,dynamic textures are image sequences that vary continuously with time.The dynamic processes of dynamic texture also differ from the optical flow field in image motion analysis,which mostly do not meet the brightness conservation and local smoothness constraints in optical flow.Therefore,in the modeling process,we need to take into account the characteristics of dynamic texture data to describe the appearance and dynamics accu-rately.The effectiveness of dynamic texture modeling depends on the associated visual tasks.It is necessary to select flexible way to model dynamic texture for different visual tasks in order to obtain the ability to solve the specific problems.In this thesis,we study some key issues of dynamic texture modeling and its appli-cation,including dynamic texture feature extraction,feature representation in recogni-tion tasks,model construction in synthesis tasks,and learning the synthesizability of dynamic texture samples.To this end,the thesis centered on dynamic texture mod-eling,and adopted a task-driven modeling strategy,which designed different modeling ways according to the varied requirements in different applications.The main research contents are as follows:(1)Aiming at the problem that the existing dynamic texture feature extraction methods are deficient in spatial geometric description,a dynamic texture feature coding method based on shape co-occurrence patterns is proposed.The level set method is used to compute the topographic map of the dynamic texture appearance,to construct the shape cascaded structure of the texture image,and extract the invariants of the shape texton in the tree of shapes as local features.We use the membership of the node in the tree of shapes to construct the adjacency graph of the texture shape textons.In the time dimension,the image sequence is projected into the shape co-occurrence pattern dictionary to obtain the implicit temporal information,and then the spatial structure information is combined to capture the spatial geometric aspects and temporal stationarity simultaneously in DTs.(2)To solve the problem that it is difficult to establish an integrated spatial-temporal joint model in dynamic texture recognition,a dynamic texture representation modeling by aggregating space-time feature has been proposed.We take into account the temporal self-similarity of dynamic texture data,and suppress temporal redundancy to simplify the description of dynamic textures.Dynamic texture feature representation is modeled from the spatial dimension and the temporal dimension respectively.We adopted the way of ensemble learning to fuse multiple features by aggregating spatial and temporal texture features.A method based on complementary feature representa-tion of appearance and dynamics has been proposed to improve the accuracy of dynamic texture recognition.(3)Due to the lack of constraint ability of the existing statistical model in dynamic texture synthesis,the synthesized dynamic textures have a tendency toward smoothing details to some extent.In this thesis,a statistical constraint model was constructed based on the correlation statistics of spatiotemporal convolutional neural network filter response.The spatial-temporal statistical features of dynamic texture were modeled and a dynamic texture synthesis method based on deep learning was developed.Addi-tionally,the impact of deep model on the dynamic texture synthesis is explored.The experiments show that the shallow network with random filters can synthesize station-ary dynamic textures without training,which indicates that the second-order statistical properties are sufficient to represent Gaussian process in stationary dynamic textures.(4)In example-based dynamic texture synthesis(EDTS),the synthesis result is subject to both the synthesis algorithms and the synthesizability of samples.The prob-lem of EDTS has been studied for several decades,but none of the existing synthesis methods are able to tackle all kinds of dynamic textures equally well.Rather than focus on new synthesis methods,we turn to another way to help EDTS by investigating dynamic texture synthesizability-how synthesizable a specific dynamic texture sample is by EDTS.Regarding whether the samples can be synthesized or not,we combine the tasks of dynamic texture recognition and synthesis.We use a data-driven approach to learn the model of relationship between dynamic texture samples and synthesizability.We propose to predict synthesizability score of a given dynamic texture sample,and suggest which EDTS method is best suited to synthesize it.The synthesizability pro-vides strong support for both sample and algorithm selection for practical applications in dynamic texture synthesis.