Research On English Textual Entailment Recognition Based On LSTM

Recognizing textual entailment(RTE) is an essential challenge in Natural Language Processing. It is widely used in many traditional NLP applications, such as Machine Translation, Automatic Summarization and Information Retrieval and so on. Many related Evaluations have greatly promoted the development of textual entailment recognition in the past ten years, and more and more researchers pay their attention to this area. Traditional methods to RTE has been the dominion of classifiers employing hand engineered features, which heavily relied on natural language processing pipelines. And the generation of features usually need a lot of e xternal semantic resources, the process is both labor intensive and time consuming.With the boom and development of deep learning technology, more and more researchers employ DL approaches on NLP tasks, and much progress has been achieved recently. End-to-end neutral networks can effectively avoid the two drawbacks mentioned above. The recently released Stanford Natural Language Inference(SNLI) corpus is two orders of magnitude larger than all other resources of its type, which made it possible to employ deep neural networks for the RTE task. So far proposed deep learning approaches for RTE can be roughly categorized into two groups: sentence encoding-based models and matching encoding-based models. And most of the proposed methods employed LSTM to encode sentences. In this paper we present several approaches for RTE, it mainly contain the following three aspects:1. Logistic regression-based model for RTE. This part of work is one of our baseline method in our paper. We generated four kinds of pattern-related and similarity-related features which make full use of the properties of SNLI. And we filtered these features to obtain most effective ones. After pattern filtration, a logistic regression model was employed for classification.2. Sentence encoding-based model for RTE. We proposed a sentence encodingbased model for RTE. This model is based on the architecture of Siamese Network, premises and hypothesis are independently encoded by bidirectional LSTM, and the classification is based on building the “matching vector” generated by three matching heuristics: concatenation, element-wise difference and product. Finally the “matching vector” is fed into Softmax for classification.3. Neural network model with Attention for RTE. Implemented a sentence matching-based model which combines the hypothesis and premise before input, and the sentences are encoded by LSTM. We employed the Attention mechanism on both of the two neural networks mentioned above. Specifically, the Attention mechanism used in sentence-encoding model is called “Inner-Attention” mechanism in our paper. Inner-Attention helps generate a more focused and accurate sentence representation by weight the words according to their importance. Experiments conducted on Stanford Natural Language Inference(SNLI) Corpus have proved the effectiveness of “Inner-Attention” mechanism. With less number of parameters, our model outperformed the existing best sentence encoding-based approach by a large margin. In addition, we proposed a voting mechanism which combi ned the results of several models, which further boosted our performance.