Research On Cross-domain Knowledge Transfer In Natural Language Understanding

Natural Language Understanding is designed to take human language as input and get a machine-readable semantic representation.It not only allows the computer to understand human language,but also helps deepen the understanding of language ability and human intelligence.Deep-neural-network-based methods have achieved good performances on multiple natural language understanding tasks,while relying on largescale annotation data.For some domains,the annotation data are difficult to obtain,or the manual labeling is very difficult.It's hard to directly obtain models that perform well on this domain without large-scale annotation data.At this time,cross-domain knowledge transfer can help alleviate the above data dependency problems by using the labeled data of other domains to help the learning of the target domain,which has great theoretical research significance and practicality.In recent years,researches on cross-domain knowledge transfer focus on two aspects: single-domain knowledge transfer and multi-domain knowledge transfer.The former aims to transfer knowledge from a source domain to the target domain,while the latter aims to transfer knowledge between domains.The common goal of both is to transfer common knowledge while maintaining the characteristics of each domain.At present,there are two problems in cross-domain knowledge transfer: In the data perspective,there is the domain relevance divergence.In the same domain,there are differences in domain characteristics,different samples and different words show different degrees of domain relevance.In the model perspective,there is the cross-domain parameter sharing problem.The domain differences are reflected in multiple levels among different domains,and it is pretty difficult to share model parameters between different domains.For the topic of cross-domain knowledge transfer in natural language understanding,in order to solve the above problems,this paper proposes a two-perspective solution of data and model.By modeling more fine-grained domain relevance and sharing model parameters at a more detailed level in the process of single-domain and multi-domain knowledge transfer,where domain general knowledge is better transferred,while maintaining the characteristics of each domain.The main contents of this paper are as follows:1.From the perspective of data,focusing on the problem of domain relevance divergence,a solution for modeling sample-level and element-level domain relevance is proposed,and the domain relevance is introduced into the process of single domain knowledge transfer to achieve knowledge fusion.Experiments are carried out on three sequence labeling tasks,and the results prove that this method effectively reduced negative transfer and enhanced the effect of knowledge transfer.2.From the perspective of the model,we focus on the cross-domain parameter sharing problem.Through the domain-related attention mechanism,we model a more detailed domain divergence,and achieve a more detailed cross-domain parameter sharing in multi-domain knowledge transfer.Experiments are conducted on machine reading comprehension and sentiment analysis tasks.The results show that this method effectively improves the performance of various domains.Compared with other multi-domain knowledge transfer methods,this method has certain advantages in parameter scale and speed.3.In order to better show the effect of the above two cross-domain knowledge transfer methods,and at the same time,implement them into specific applications.Taking the Chinese word segmentation task as an example,this paper constructs Chinese word segmentation cross-domain knowledge transfer display system,which provides word segmentation tools for multiple domains,and show sthe effect of crossdomain knowledge transfer in multiple domains.