Research On Deep Learning Based Textual Entailment Techniques

The reasoning relationship in natural language,also known as textual entailment,is a fundamental relation between texts,widely distributed in natural language texts.Many information processing tasks have to face the texts including textual entailment.If there is a technique that could handle the entailment relationship,it will help the processing of those tasks.Therefore,research related to textual entailment is an essential work in the natural language processing area,which could assist other information processing tasks,and has abundant application scenarios.In general,a binary relationship is often studied in three perspectives,i.e.,the recognition,extraction and generating of the relation.As a binary relationship,textual entailment also has three basic research tasks,i.e.,recognizing textual entailment(also known as natural language inference,NLI),mining entailment knowledge,generating entailment hypothesis.Although made some impressive progress,previous works also had some limitations.Firstly,the previous textual entailment recognizing models are complicated and hard to train;secondly,natural language inference approaches could not involve knowledge effectively;thirdly,they could not balance the accuracy and coverage of mined entailment knowledge;lastly,the generated hypotheses could not keep important information,and an objective measurement is also absent.To address these four shortcomings in previous works on the three basic tasks,we proposed four models respectively,which all obtained state-of-the-art performance on publicly available datasets.The main research contents are listed as follows:Firstly,we researched recognizing textual entailment using Gaussian Transformer.Current neural NLI approaches could be categorized into three types: models based on recurrent neural networks(RNNs),convolutional neural networks(CNNs),and selfattention networks(SANs).Although obtaining impressive performance,previous recurrent approaches are hard to handle tokens in sentences in parallel,which slows down the training stage;convolutional models tend to cost more parameters;self-attention networks are not good at capturing local dependency of texts.To address these problems,we propose an efficient RNN/CNN-free architecture named Gaussian Transformer.Experiments show our model achieves new state-of-the-art performance on public benchmarks.Secondly,we explored natural language inference using evidence from knowledgegraphs.For human beings,knowledge plays an essential role when performing inference.Although obtained impressive performance on standard NLI benchmarks,previous methods encounter performance degradation when being applied to some specialized,knowledge-intensive areas,such as the medical domain.To fill the lack of knowledge,we propose a simple Evidence-Based Inference Model(EBIM)using knowledge graphs,which could employ the relevant evidence from knowledge graphs to improve the inference performance on those areas.Experiments on the science question answering corpus,i.e.,Sci Tail,show that the proposed approach performs better than other external knowledgebased methods;evaluation on the medical NLI dataset,i.e.,Med NLI,demonstrates that the proposed EBIM method achieved state-of-the-art accuracy.Thirdly,we studied predicate-based entailment rules mining using deep contextual architecture.The knowledge about inference is often represented in the form of predicatebased entailment rules.Many efforts have been dedicated to extracting predicate-based entailment rules from text corpora by utilizing statistical methodology including distributional hypothesis and Latent Dirichlet Allocation(LDA).However,these studies could not give equal consideration to both coverage and accuracy of the mined rules,which brings instability to downstream applications.To solve this problem,we proposed a novel model named Deep Contextual Architecture(DCA).Combining benefits from both statistical information and semantic representation,the proposed DCA model shows the potential for better modeling the context of predicates.Evaluation on public datasets demonstrates that our method outperforms several strong baselines.Lastly,we investigated generating textual entailment using residual long-short-term memory networks.Current sequence-to-sequence GTE models are prone to forget information from premise and not good at handling complex premises.Moreover,the lack of appropriate evaluation criteria,which should capture the diversity of hypothesis,hinders researches on GTE.To resolve these two issues,we propose a residual long-short-term memory networks based model and a novel metric for GTE,namely EBR(Evaluated By Recognizing textual entailment).Experimental results show that our model achieved state-of-the-art performance,and EBR metric also overcame the defects of current criteria,showing high consistency with human evaluation.