Research On Machine Comprehension Technology Based On Attention Mechanism

Machine reading comprehension task is an important sub-task in the field of natural language processing,and also one of the most important supporting technology of automatic question answering.The complexity of machine reading comprehension makes it an important aspect for evaluating the natural language comprehension ability of machine.At the same time,in recent years,with the rapid development of the Internet era,a large number of natural texts have been accumulated on the Internet.How to use the massive text resources to read the text,and to be able to complete a more direct question and answer,is also a subject with practical application value.Benefited from the development of deep learning technology and the release of more challenging and practical datasets,the heat and challenge of machine reading comprehension are rising.Since the machine reading comprehension task mainly solves the reasoning answer of long passage,and the attention mechanism can provide a flexible and effective way of information interaction and utilization,it has become an indispensable key technology module in the machine reading comprehension task.Therefore,the research on machine reading comprehension technology which is based on attention mechanism,has important theoretical value and broad application prospects.In this paper,the attention-based neural model technology in machine reading comprehension task is taken as the main research object,and various attention mechanisms are carried out on the selection and application of various attention mechanisms in different model architectures,attention concerns at different levels and granularity,and attention modeling from different perspectives.The main research work of this paper is as follows:Firstly,the basic model is analyzed and explored.This chapter introduces the task background and definition of machine reading comprehension,and introduces the related technologies used in this paper.Then,the construction method of machine reading comprehension model based on various deep neural networks is studied.According to the characteristics of different encoding methods of deep neural network in machine reading comprehension task,experiments are conducted to compare various attention modes and assistant means,so as to analyze the role of key technologies in the model and the end-to-end of different designs.The performance of end-model in accuracy and time consumption lays a foundation for the improvement and analysis of machine reading comprehension tasks.Secondly,on this basis,the method of connecting hierarchical information in machine reading comprehension is studied.Different from the previous model,which only focuses on the middle part,the model will calculate the problems and texts at different levels in many places,and retain the historical information for subsequent attention modules,so as to enable the interaction of hierarchical information between models.In order to reduce the information loss after multi-layer transmission,a cross-layer transmission method is used to transfer the attended representation.At the same time,a gated method is introduced to selectively control the transmission of information.Finally,a machine reading comprehension model based on position attention mechanism is proposed,and the experimental improvement and analysis are carried out.Different from the semantics-based attention modes of previous chapter,the model is designed based on prior knowledge in the process of answering in human reading comprehension.Different kernel functions are used to describe the prior hypothesis in machine reading comprehension tasks,and external knowledge is used to improve the utilization of position information.Then attention mechanism is used to interact position angles.Finally,in order to compare the different ways of introducing position information,some other methods,such as position encoder,are used for control experiment.The experimental results show that this priori hypothesis is consistent with the task background,and the attention mechanism can make the model better use of position information and help answer questions.