Research On Machine Reading Comprehension Model Based On Multi-hierarchy Self-attention Mechanism

Machine reading comprehension is one of the research hotspots in the field of natural language processing.It is also a long-term goal of artificial intelligence in the process of processing and understanding human language.The main content of machine reading comprehension research is to give text and questions,and let the machine answer questions according to the content and meaning of the text.With the development of artificial intelligence,using the deep neural network model to solve the task of machine reading comprehension has become the mainstream.The existing machine reading comprehension model can be divided into two parts: text representation and answer selection.In the text representation part,most of the models are based on LSTM or GRU.However,due to the continuous sequence learning characteristics of the RNN network,the semantic information extracted by the model disappears due to the long-distance Association,and the model is relatively time-consuming in training.In order to solve the problem of long-distance semantic information disappearing,many models introduce attention mechanism to integrate RNN network.However,the application of attention mechanism in many models is not detailed enough,and the effect of using attention mechanism in text representation at multiple semantic hierarchies is not considered.In view of the above problems,this paper studies the method of text representation using attention mechanism at multiple hierarchies.The main study of this article is as below:1)A machine reading comprehension model MHS based on Multi-Hierarchy Self-attention mechanism is proposed.According to the hierarchical structure of the text,the model divides the text into several hierarchies,such as the relevance between multiple independent sentences,the relevance between words within the sentence and the relevance between words from different sentences.At these hierarchies,only the self-attention mechanism is used to extract and represent the semantic information of the text.Through the test and experiment on the Stanford machine reading comprehension data set squad1.1,MHS gets the results of EM value 71.2% and F1 value 81.5%,and the above performance indexes are better than the RNN network model.In the late stage of Stanford's squad2.0 data set,MHS achieved the EM value of 68.3% and F1 value of 70.1%,which is better than the baseline model Doc QA and BNA.2)The stability of MHS model is studied.Comparing the stability of MHS and RNN based models,the range of F1 value of MHS model is within 1.7%,while that of RNN based model is within 5.9%.The experimental results show that the stability of MHS is better than that of RNN based model,and the generalization ability of MHS is stronger in data sets with large data changes.3)In this paper,we study the influence of the connection between hierarchies on the effect of the model.In this paper,an improved connection method based on soft threshold function is introduced into the multi-layer self-attention mechanism,and a comparative experiment with the commonly used connection method,densenet and RESNET,on the squad1.1 data set is carried out.Using the improved connection method based on soft threshold function,compared with densenet,the F1 value of the model is increased by 4.7%,compared with RESNET,the F1 value of the model is increased by 1.8%.The above experimental results show that the interlayer connection based on soft threshold function can improve the effect of MHS in answering questions on data sets.