Research On Interpretable Variational Autoencoder With Concept Embedded

Variational autoencoder obtains good representation by learning the feature mapping between latent space and data space.However,unsupervised training process of VAE turns it into black-box model.Latent space learned by the model lacks interpretability,which greatly limits the development of the model.Therefore,how to improve the interpretability of variational autoencoder has become an urgent problem to be solved.The variants of variational autoencoder mainly improve the interpretability of latent space by optimizing the prior and posterior distributions of the models.On the one hand,variational autoencoder constrains the construction of latent space through prior distributions.However,raw data is composed of one or more concepts,and its latent space is relatively complex.The existing variational autoencoder and its variants obey prior distributions that are complex in structure,artificially hypothetical and unable to describe the randomness of latent variables,making them unable to describe the concept of the latent space and guide them to explain the aggregation process of latent space.On the other hand,existing methods impose regularization on the posterior distribution to constrain the latent space to learn interpretable latent representations.However,these methods obey the standard normal prior distribution which cannot learn the different characteristics of concepts in latent space.In order to solve these problems,this thesis studied Interpretable Variational Autoencoder with Concept Embedded,and investigates the interpretability of the latent space of variational autoencoder from three aspects:the concept representation method of the prior distribution,concept embedded variational autoencoder,and the latent representation of different concepts under the latent space,respectively.The main research contents and innovations of this thesis are as follows:(1)In order to solve the problem that variational autoencoder obeys the prior distributions that are complex and unable to describe the randomness of the latent variables,Cloud-Cluster based Concept Learning of Latent Space(Cloud-Cluster-CL)is proposed.First,Cloud Model based Clustering Algorithm(Cloud-Cluster)is designed to introduce hyper-entropy into the data to describe randomness and combine the data-to-concept uncertainty with an expectations-based multi-step backward cloud transformation algorithm converts to obtain the optimal concept representation.Then,concept learning method based on latent space of autoencoder using Cloud-Cluster to obtain multiple coarse concept representations in latent space as the prior distribution of variational autoencoder.Cloud-Cluster has higher clustering performance on latent space of autoencoder and better clustering internal metrics results on MNIST,Fashion MNIST and USPS datasets.In addition,experimental results from UCI and OpenML clustering datasets show that Cloud-Cluster improves the average clustering accuracy by 9.5%,42.9%and 10.3%compared to uncertainty theory-based clustering algorithms such as FCM,PFCM and GMM,respectively.(2)In order to solve the problem that the construction process of latent space of variational autoencoder under complex prior distribution is complicated and lacks comprehensibility,Coarse Concept Embedded Variational Autoencoder(C2-VAE)is proposed.Firstly,the encoder is used to encode concept parameters of latent variables,and the encoded hyper-entropy is introduced into the reparameterization process of the model to increase the randomness of latent variables,which expands the sampling range of variables,improves the flexibility of data,and represents the latent space information more accurately.Then,variational inference with concept embedded is proposed,which embeds multiple latent space coarse concepts under Cloud-Cluster-CL into variational autoencoder as an effective prior distribution to constrain model,and a theoretical derivation of variational lower bound with coarse concept embedded is combined with variation method to guide the model training to obtain the optimal parameter estimates.It realizes the coarse concept parameter update and the mutual mapping between latent space and concept space.Finally,experimental results on MNIST,Fashion MNSIT and USPS datasets show that C2-VAE improves the clustering performance indicator NMI by 22.9%and 19.9%compared with two deep clustering methods,VaDE and GMVAE,respectively,showing better clustering and reconstruction performance.In addition,compared with FactorVAE,JointVAE and Guided-VAE,C2-VAE explicitly explains the aggregation process of the model,and other interpretable latent representations are found on top of the existed.(3)In order to solve the problem that the existing posterior distribution optimization methods only characterize the common features of latent variables in the whole latent space but ignore the differences between different latent variables,Coarse Concepts based Variational Autoencoder with Multiple Decoders(C2-VAE+)is proposed on the basis of C2-VAE.Firstly,the uncertainties of data into multiple coarse concepts in latent space of C2-VAE are combined to divide data into multiple sets.For the problem of fewer data after division,C2-VAE+generates new data conforming to coarse concepts by using the reparameterization method of C2-VAE.Then,coarse concepts based variational inference method of multiple decoders is proposed.Multiple sets of data are encoded by the same encoder,and multiple sets of latent variables are decoded by multiple decoders.On the basis of this,a theoretical derivation of variational lower bound of multiple decodes is combined with variation method to obtain the optimal model parameter estimates.It approximates the posterior distribution and constructs multiple sets of latent spaces,and learns the similarity between different concepts and the latent representation of latent spaces.Finally,experiments on MNIST,Fashion MNSIT,and USPS datasets show that C2-VAE+can improve the prediction performance of VAE while discovering similarities between different coarse concepts.Furthermore,compared to FactorVAE,JointVAE,and Guided-VAE methods,C2VAE+found not only existing common features,but also the different features of latent variables under different concepts.