Research On Semantic Relation Classification Based On LSTM

Recently, the NLP community has shown a renewed interest in deeper semantic analyses, among them automatic recognition of semantic relations between pairs of words. This is an important task with many potential applications in Information Retrieval, Text Summarization, Machine Translation, Question Answering, Knowledge Base Population, Word Sense Disambiguation and Language Modelling.In NLP community, breakthroughs have been made by deep learning on a variety of tasks and domains such as language modeling, machine translation and semantic information retrieval. Therefore, there is some work about applying deep learning techniques to the task of classification of semantic relations. But we think the potential of deep learning on this task has not been fully explored. Long Short-Term Memory(LSTM) is a specific recurrent neural network(RNN) architecture that was designed to model temporal sequences and their long-range dependencies more accurately than conventional RNNs. Recently research has shown that LSTM RNNs are well suited for modeling sequence data like text. Therefore, we propose a LSTM-based model for the task of classification of semantic relations and achieve state-of-the-art result which shows the ability of LSTM to capture semantic information in sequence text.In the stage of pre-processing and feature extraction, we propose the relative dependency feature. After preprocessing and feature extraction, all data of samples will be translated to real vectors(embedding) and pass through a Bi-LSTM(BLSTM) layer followed by a relative max pooling which will produce the sentence level feature of samples. The lexical feature will extracted from output of embedding layer and BLSTM layer. A multilayer perceptron(MLP) will be used for combining sentence level feature and lexical feature into the final extracted feature vector. Finally, the final extracted features are feed into a softmax classifier to predict the sematic relation labels.What’s more, we have investigated many deep learning methods and tricks during the model building and training process. Some empirical evaluation will be made based on all experimental observations.