Attribute Control Via Adversarial Learning In Natural Language Generation

Language is an important tool for human's information expression and communication.By using language,people can describe some objective facts or express their subjective thoughts.With the rapid development of deep learning techniques,Natural Language Generation(NLG),a core area of Natural Language Processing and Artificial Intelligence,is attracting more and more academic attention.There are also many related industrial applications,such as machine writing and chatbots.Flexibility and diversity are significant features of human language expression.To express the same or similar semantic contents,human can flexibly choose diverse forms.During the organization of language,people are not only aware of what semantic contents to describe,but also affected by some attributes,which are independent of contents,such as linguistic style and sentiment.To achieve better performance in real scenarios,when generating text results,NLG models or systems should not only ensure the accurate expression of desired contents,but also take control of the final form to make it adapted to specific scenarios.Therefore,more and more researchers have begun to study the method of attribute control in language generation.Natural language generation with attribute control requires not only the accurate control of generated results in specific attributes,but also the semantic consistency between outputs and desired target.Most existing approaches have satisfied the previous requirement,but they often produce some texts which are seriously irrelevant to the desired semantic content.To tackle this problem,this thesis analyzes the basic methods and recent advances of natural language generation with attribute control,and then points out that their main common drawback is the lack of direct supervision of the task-specific semantic relevance between the input and output texts.Based on adversarial learning,this thesis proposes to introduce some discriminative models,which are related to text semantic relevance,to guide the learning of attribute-controllable NLG models.For text modification and dialogue generaton,this thesis designs different task-specific solutions,and the main contributions can be summarized as follows:1.For the task of text attribute transfer,this thesis proposes a novel unsupervised adversarial text attibute transfer model which is based on a proposed original partial comparisions.At first,we mine paired samples from the non-parallel corpora in terms of attribute and attribute-independent semantic content with a rule-based method.Then we employ two neural sentence pair models to learn pairwise characteristics from the obtained data.Finally,these discriminative models are used to guide the learning of text attribute transfer models by adversarial training.Experiment results show that the proposed approach can achieve better performance than most existing models on content preservation.2.For the task of response generation with atttribute control,existing models tends to generate some ‘general responses',which have low semantic relevance to input utterances.Different from text attribute transfer,there exists large-scale parallel corpora in most controllable dialogue generation scenarios.To solve the problem of ‘general responses',this thesis proposes a novel model learning framework,which bridges controllable dialogue generation with parallel corpora and text attribute transfer with only non-parallel corpora via adversarial learning and multitask learning for better performance,especially the enhancement of dialog semantic relevance.Experiment results show that the proposed approach can achieve not only accurate attribute control but also higher semantic relevance and generation diversity.