Generating Diverse Texts Via Reinforced Variational Networks

Enabling computers with the capability of generating texts is one of the critical goals of realizing Artificial Intelligence.It is also a current research hotspot in the field of Natural Language Processing.In recent years,deep learning has become the mainstream method of text generation,which makes the generated texts more fluent and greatly improves the qualities of the generated texts.However,mainstream deep learning methods of text generation are based on Sequence-to-Sequence models,which always generate the constant target texts for the same source text.It is not suitable for the application scenarios that require diverse responses,such as chat robots.Therefore,improving the diversities of the generated texts under the premise of satisfying high quality has become an important research topic for text generation tasks.To make the generated texts have diversities,researchers have proposed methods based on variational encoder-decoder,which have achieved encouraging results.These methods encode a source text into a multivariate normal distribution and use the real-valued vectors sampled from the multivariate normal distribution to generate texts.Although these kinds of variational encoder-decoder methods produce texts with certain diversities,they use cross-entropy as the loss function to make the decoder fit the surface expression of the annotated text,resulting in the generated texts have only certain surface diversities,and there is still a lack of sufficient diversities in semantics.Besides,the decoder side of the existing methods adopts a sequence model,and the Teacher Forcing mechanism in the training process can easily lead to exposure bias,which reduces the relevance of the generated texts.In response to the above problems,this paper has carried out the following work:First,aiming at solving a lack of semantic diversities in the generated texts,this paper proposes a model with a Target-Side Variational Network(TSVN).The proposed model uses TSVN to capture the diverse semantics of annotated text and alleviates the issue that the generated texts have only surface diversities.Besides,a tensor network is built on the encoder side to better capture the semantics of different aspects in the source text.This paper performs the experiments on question generation task and dialogue generation task and evaluates the proposed method from the perspectives of relevance and diversity.The experimental results show that although the relevance of the generated texts slightly decreases,the diversity is improved.Second,aiming at alleviating the problem that the relevance of the generated texts decreases in the first part of the work,this paper proposes a reinforcement learning method to improve the relevance of the generated texts.This method regards the model proposed in the first part as an agent and builds a relevance scorer to measure the relevance of the generated texts,then it adopts the relevance score as a reward and optimizes the policy parameters according to the reward by using the policy gradient method,which guides the agent to generate texts with more relevance.The experimental results show that the proposed reinforcement learning method improves the relevance of the generated texts while basically maintaining diversity.