Research On Text Generation Models Based On Latent Variables

Text generation is aiming to automatically generate or extract normative and language compliant text,which is a great challenge in the field of natural language processing.The technology of natural language processing often belongs to the exploration on the cognitive field.In general,the understanding of language requires contextual scenarios or a large amount of prior knowledge as references.With the development of language models,text generation models are often modeled as sequence-to-sequence Encoder-Decoder models.Text generation models are mainly divided into generative models and extractive models.And the typical generative models suffer from the paradigm of the maximum likelihood estimation method,which predict current word relying on the previous generated words.However,most of generation models ignore the influence of unobservable or missing data,which mean ignore the importance of the latent variables.In this thesis,we explore the text generation models of generative models and the extractive models through the exploration of the latent variable models.The main research works are as follows:(1)As for generative models,the thesis proposes a novel framework of the task of text generation called Encoder-Decoder-Discriminator architecture by formally modeling text generation models with the study of generative adversarial networks.And through the study of variational auto-encoders,it proposes LatentGAN model based on latent variables for text generation.The introduced latent variables and the assumption of posterior distribution help to capture the distribution of potential features in the text and to solve the problem of less diversity in generative adversarial networks.It introduces adversarially training process for alleviating the problem of low quality of generated text in variational auto-encoder models.The LatentGAN model is designed and implemented,where the generator is modeled with the long short term memory network and the convolutional neural network model is designed as the discriminator for binary classifying.It also introduces the idea of defining the reward function in reinforcement learning to define the loss function of the generator.Experiments conducted on the dataset of Chinese poem demonstrate the effectiveness of the proposed model compared to other methods.(2)As for extractive models,the relation extraction task of natural language processing is modeled as an extractive text generation model based on latent variables(relations).And it proposes TDRE model,which is a relation extraction method based on tensor decomposition.The model models the extracted triplets in the form of tensor,which can handle multi-label entity-pairs and solve the problem of overlapping relation labels.And we propose a relation extraction model based on DEDICOM tensor decomposition algorithms,which decomposes in the relational component for obtaining the internal relationships between relation types.In this thesis,the model of conditional random field is used in the module of entity recognition,bi-directional long short-term memory network is used for relation classifying and DEDICOM strategy is used for relation extraction.Experiments conducted on the datasets of NYT10 dataset,CoNLL04 dataset and ADE dataset have demonstrated that our models are better than the current state-of-the-art models,which proves the effectiveness of the tensor decomposition algorithm based on the relational dimension.