| This thesis aims to study visual generative models from both theoretical and applied perspectives,and demonstrate the results obtained through the research.The main objective of our study is to achieve efficient,effective,controllable,and dynamic visual data generation with various types of generative models.The article is structured in four sections that follow a progressive narrative: first,a summary of commonly-used generative models in visual generation on their concepts and background knowledge,including energy-based models,variational autoencoders,and generative adversarial networks.Next,this article will follow the time development order of those models;and respectively introduce three elements — ”particles”,”causality”,and ”temporality” — that have important influences on the theory and application of visual generation;then conduct experiment and discussion on this topic.Among the three elements,particles refer to dynamic particles,which come from the physical simulations on the parameter estimation of energy-based model through iterative generation.The sampled data continuously evolve in the functional space defined by parameters,just like countless high-energy particles moving in Brownian motion.When energy dissipates,dynamic particles return to a steady state,and the model converges.At this time,disorderly data samples become ordered and conform to the approximation of the real distribution.Based on this theory,we found and proposed the learning of energy models from the particle perspective,and took this opportunity to introduce the optimal transport theory of dynamic particles to optimize and accelerate the training of the model.As for the generative models of non-sampling iterative learning,namely the implicit generative models,we focus more on studying their intrinsic causal relationships.The problem with implicit generative models is that their generated visual data are uncontrollable.Most existing research uses various auxiliary conditions to enhance the relationship between annotation information and latent variables to achieve controllability.However,in order to accomplish this goal,due to the influence of exogenous factors,multiple auxiliary conditions must be utilized for enhancing the correlation.This approach requires more complex model architecture,higher computing power requirements,and more unstable model training and testing,which is contrary to the original intention.To solve these problems,we propose to use only a single knowledge graph as a condition for controllable generation.Through causal analysis and the establishment of a causal diagram model,we propose a graph-to-image generation theory based on consensus learning.This theory was later applied and experimentally verified feasible,and achieving improvement in effect and performance compared with existing methods.At the same time,to further achieve generative controllability,we added a visual generation task,which raised the data dimensionality to dynamic temporal dimensionality,and conducted research on controllable video generation tasks,exploring the impact of the ”temporality” element on generation.Of course,in this part,since the development of the video generation is still in the early stages compared to image and lowdimensional data generation,our research progress and contributions are relatively limited and can be summarized into three aspects: first,innovation: we are the first to propose a multi-object controllable video generation task;second,technology: we propose a generative framework for video synthesis based on single-frame object layout as a condition for multi-object controllability;third,verification: the feasibility,effectiveness,and most importantly,controllability of the model and generation framework were verified through various indicators and methods.In conclusion,we focus on the task of visual generation and explore various generative models from the most concerned perspectives of controllability and effectiveness.Innovative methods and task modes are proposed from multiple research and theoretical perspectives,and the research results are validated through final applied experiments. |
