Research On Transfer Of Dependency Parsing Based On Lexical-level Knowledge

With the development of natural language processing,dependency parsing has been paid much more attention as an important fundamental task.Dependency parsing aims to analyze the modified relationships between words within a sentence and can give syntactic guidance in downstream NLP tasks such as machine translation,information extraction,question answering and so on.Statistical and neural-based dependency parsing rely on treebank,otherwise there will be problems of overfitting,leading to unsatisfied perfor-mance in low-resource domains.Treebank annotation needs much linguistic knowledge,especially it is very expensive and time-consuming.As a result,research on how to transfer existing syntactic data into low-resource domain is very valuable.During the process of data transferring,different knowledge forms and huge lexical bias are the main obstacles which many impact the performance.In order to improve the transfer performance,this thesis uses four kinds of lexical knowledge to bridge the gap of data among different domains.In detail,this thesis proposes to extract syntactic knowledge from query logs based on co-occurrence to improve the performance of unsu-pervised dependency parsing,particularly using syntactic clusters,domain invariant word embeddings,masking domain specific elements to alleviate the lexical bias problem in domain transfer.1.Unsupervised dependency parsing based on knowledge of lexical co-occurrence.Recently,unsupervised dependency parsing has been quite popular because it does not need any expensive treebank.However,its accuracy is too low to be used in practice,partly due to the fact that their parsing model is not sufficient to capture linguistic phenomena underlying texts and lacks syntactic knowledge.The performance for unsupervised dependency parsing can be improved by incorporating syntactic knowl-edge extracted from texts.Syntactic knowledge is acquired from query logs to evaluate the dependency relations between two words through a score function.We construct query-augmented DMV(QA-DMV)which uses syntactic knowledge from query to help estimate better probabilities in dependency model with valence.Our method is language independent and obtains big improvements in Chinese and English datasets.Moreover,experiments show that the proposed model achieves bigger improvements with more query logs.2.Dependency parsing for biomedical text based on syntactic clusters.Tree-banks for biomedical text are not enough for supervised dependency parsing either in their scale or diversity,leading to data sparseness problem.Discrete symbolic features are very common in statistical machine learning and they are the key solutions for data sparseness.We propose to cluster words according to their dependency-based word em-bedding.Words with similar syntactic roles will be within one cluster.And we use syntactic clusters as features in statistical dependency parser and use dependency-based word embeddings as the input of neural-based dependency parser.Experiments show that syntactic cluster can capture syntactic properties of words and syntactic clusters can im-prove performance of biomedical dependency parser.Syntactic clusters can also be used to combine with brown cluster which can make bigger performance improvements.More-over,dependency-based word embedding improves the parsing accuracy when the data from news domain is transferred into biomedical domain in a neural-based dependency parser.3.Domain adaptation for dependency parsing based on domain invariant word representations.The mainstream method of adapting dependency data from one resource-rich domain to resource-poor domain is to find their shared feature set.For neural network models,word embeddings are fundamental features.However,little work has been done in investigating domain invariant word embeddings.Pretrained word embeddings are trained on general domains and they perform not very well in domain adaptation of specific data.To promote data exchange between source and target domains,this paper propose to learn domain-invariant word representations by fine-tuning pretrained word representations adversarially using data of two domains.Experiments of dependency adaptation between biomedical and news wire domain show the effectiveness of domain invariant word representations in alleviating lexical bias between source and target data.4.Partial delexicalized dependency parsing based on lexical domain property.Through analyzing many data,we find that there is a big lexical gap among source and target domains especially in noun and adjective words.This gap makes it difficult to transfer syntactic data between source and target domains.To address this issue,we consider contextualized domain property for each token or word using generative adversarial network in format of mask sequence for each sentence.Masks for domain specific elements are used in both implicit and explicit methods.Implicit application is to concatenate the representations of original word and[MASK].Explicit application is to substitute original words with[MASK]or other new words with different strategies and new data are used as augmented data.Experimental results show that both implicit and explicit methods can transform the data of source domain to target domain more effectively.