Research On Domain Adaptation For Dependency Parsing

Dependency parsing derives the syntactic and semantic information between two words of the input sentence via a dependency tree.The dependency tree is a tree structure composed of input words,where the directed edge from the head word to the modifier word is a dependency arc,and the label above the arc represents the dependency relation type.As a fundamental natural language processing task,dependency parsing has attracted more attentions of researchers due to its simplicity and comprehensibility.The parsing results not only promote the development of natural language processing tasks,such as word segmentation,semantic role labeling,etc,but also provide supports for other artificial intelligence tasks,such as machine translation,information retrieval,etc.In recent years,supervised dependency parsing model has obviously improved in the efficiency and performance.However,training an efficient model always relies on sufficient in-domain training data.Once the training data is from the out-of-domain,the parsing accuracy degrades significantly.The main reason is that the lexical,syntactic and semantic changes in training data from different domains make it difficult for the model to capture the commonalities and discrepancies between the domain feature distributions.Hence,how to effectively model the domain-invariant and domain-specific features becomes the key challenge for domain adaptation for dependency parsing.To address this problem,this paper first attempts to build a dependency parsing model as the foundation of the following domain adaptation researches.Then,this paper proposes two methods to deal with the problem of few-shot domain adaptation for dependency parsing from two aspects.Among them,the improved contextualized word representation method focuses on enhancing the capability of extracting domain-invariant features,whereas the method based dynamic feature transformation pays more attention on excavating the in-depth relevance between domain-specific representations.Finally,this paper proposes an adversarial and parameter generation networks to deal with the problem of zero-shot domain adaptation for dependency parsing,which can extract domain-invariant representations and fuse multiple domain-specific ones simultaneously.1.Constructing a basic model for dependency parsing to provide supports for domain adaptation researches.The current widely used dependency parsing approaches employ BiLSTM to encode input sentences.Motivated by previous works,this work for the first time applies the self-attention mechanism to dependency parsing,leading to competitive performance on both English and Chinese benchmark data.Based on the detailed analysis,we then find that combining the power of both BiLSTM and self-attention via model ensembles can further improve the parsing performance.Finally,this paper explores the recently proposed contextualized word representations as extra input features,achieving new state-of-the-art results.2.Proposing a domain-aware word representation approach to alleviate the interference of lexical changes on few-shot domain adaptation for dependency parsing.In this work,this paper proposes to obtain the domain-aware word representations via adversarial learning and fine-tuning BERT processes.First,this paper applies adversarial learning to three typical few-shot domain adaption methods,i.e.,parameter sharing,feature augmentation,and domain embedding with two useful strategies,i.e.,fused target-domain word representations and orthogonality constraints,thus enabling to model more pure yet effective domain-invariant representations.Simultaneously,this paper utilizes a large-scale targetdomain unlabeled data to fine-tune BERT,thus obtaining reliable contextualized word representations.Experiments on the benchmark datasets show that our proposed adversarial approaches achieve consistent improvements,and fine-tuning BERT further boosts the parsing accuracy by a large margin.3.Proposing a shared-private model based on the dynamic feature matching to alleviate the performance degradation problem of few-shot domain adaptation for dependency parsing caused by the syntactic changes.Prior work pays more attention to extracting domaininvariant information,but ignores the relevance of domain-specific ones.To address this problem,this paper designs a simple yet effective dynamic matching network to learn useful knowledge from source domain to improve the parsing accuracy of target domain.Concretely,the dynamic matching network learns appropriate matching weights automatically via mimicking well-trained source features,which can emphasize useful source domain knowledge and ignore the useless or even harmful ones.In addition,this paper designs a new training strategy to improve the capability of matching network.Experimental results show that our proposed model consistently outperforms various baselines,leading to new state-of-the-art results on all domains.4.Proposing adversarial and parameter generation networks to improve the parsing accuracy of zero-shot domain adaptation for dependency parsing via fusing multiple domain features.In this work,this paper proposes a novel model for zero-shot domain adaptation for dependency parsing.The model consists of two components,i.e.,a parameter generation network for distinguishing domain-specific features,and an adversarial network for learning domain-invariant representations.Experimental results show that our model can improve the performance of zero-shot dependency parsing performance significantly.The analysis on different domain representation strategies demonstrates that our designed distributed domain embedding can accurately capture domain gaps that benefits for the model to learn useful domain features.In conclusion,this thesis thoroughly studies on the problems of constructing the basic dependency parsing model,few-shot domain adaptation for dependency parsing,and zeroshot domain adaptation for dependency parsing.We hope that these research results can further motivate the progress of domain adaptation on other tasks.