Research On Chinese Machine Reading Comprehension Method Based On Attention Mechanism

Machine reading comprehension requires the computer to answer questions according to the given context.It is a task that can measure the machine's understanding of human language.With the development of deep learning technology,the research on machine reading comprehension has made great progress.Especially in the English field,many large-scale and high-quality datasets have been released in recent years,making many classic models based on deep learning perform very well in English machine reading comprehension tasks.However,the research in the field of Chinese started relatively late,and the progress was relatively slow.Due to the differences of language characteristics,some classic models that perform well on English datasets can not achieve the same effect when processing Chinese text,such as the Bi DAF model.Therefore,this paper studies the Chinese machine reading comprehension model based on the Bi DAF model,combines the pre-trained language model with the Bi DAF model structure,and adds a hybrid attention mechanism to further improve the model's ability to understand Chinese text.The main work is as follows:(1)The machine reading comprehension method based on pre-trained language model is studied.In order to strengthen the model's ability to represent Chinese text,combined with the structure of the Bi DAF model,an improved Chinese machine reading comprehension model based on Ro BERTa is proposed.Firstly,the model uses the pre-trained language model Ro BERTa as the encoding layer;secondly,it is modeled by Bi LSTM to alleviate the problem of insufficient ability of the pre-trained language model to capture local information;then the bi-directional attention flow structure in Bi DAF is used to further deepen the interaction between the articles and the questions.The experimental results on the CMRC2018 dataset show that the model combined with the pre-trained model performs better,and the improved model proposed in this paper is better than the machine reading comprehension model based solely on the pre-trained language model.(2)On the basis of combining the pre-trained language model Ro BERTa,in order to further improve the processing ability of the model for Chinese,a Chinese machine reading comprehension model based on the hybrid attention mechanism is proposed.Firstly,the pre-trained model Ro BERTa is used to obtain the representation at the encoding layer,and Bi LSTM is used to capture local dependencies;secondly,it is processed by the hybrid attention layer composed of two variants of self attention mechanism to learn the deep semantic representation;then combined with multiple fusion mechanism,multi-level and richer feature representation is obtained;use two-layer Bi LSTM to model,and finally send to the output layer to predict the start and end positions of the answer.Several groups of experiments are carried out on the CMRC2018 dataset and the DRCD dataset.The results show that the EM score and F1 score of the proposed model are significantly improved compared with the reproduced baseline model,which verifies the effectiveness of the model.Finally,a Chinese machine reading comprehension system is designed,and the model of this paper is applied in it to intuitively show the validity and necessity of the research content.