Research On Reasoning Method Based On Transformer

Question answering system is an important research direction in the field of natural language processing,and it is a question answering information retrieval technology.Compared with the traditional information retrieval technology,the users in the question answering system directly ask questions in the way of natural language,the system will be concise and accurate answers directly back to the users,improve the efficiency of obtaining information,in the current Internet massive data background,research question answering system has important value and significance.This paper mainly studies the reasoning method in question answering system.The traditional answer extraction method based on keyword matching and fuzzy query has poor efficiency and low accuracy,which can not meet the needs of current users.Improving knowledge reasoning technology is an important method for answer extraction in question answering system,so this paper focuses on knowledge reasoning method in question answering system.With the wide application of deep learning technology in the field of natural language processing,remarkable results have been achieved.Therefore,the application of deep learning neural network to knowledge reasoning has become an important research direction at present,and certain achievements have been achieved.However,when using deep learning to research inference methods,there are some problems such as insufficient corpus,semantic understanding deviation caused by different phonetic synonyms of Chinese pinyin,and low efficiency of inference algorithm.In order to solve these problems,this paper proposes to integrate Unified Transformer model into knowledge reasoning problem for in-depth research.The existing data enhancement model based on bidirectional cyclic neural network is improved to solve the difficulty of data enhancement task at present.And put forward the reasoning method based on Transformer model.The main research work and innovations of this paper are as follows:(1)In order to solve the problem of insufficient corpus in the knowledge reasoning of intelligent question answering system and the problem of semantic understanding deviation caused by the different pronunciation of Chinese pinyin,a data enhancement method based on SimBERT model and random pinyin replacement is proposed to enhance the data of corpus.Firstly,a tongpinyin dictionary is constructed to replace random words in the corpus and generate new sentences to increase the semantic diversity of sentences.Then,SimBERT model is used for data enhancement to further enrich the corpus without changing the semantics and retaining entities.The results show that the distinct-1 and DISTINCt-2 values of DuConv and KdConv data sets using the data enhancement method proposed in this study reach 0.166 and 0.680,respectively.The distinct-1 and DISTINCt-2 values of THE KdConv dataset using the proposed data enhancement method reached 0.203 and 0.721,respectively.The accuracy rate,recall rate and F1 value of DuConv data set are 91.37%,88.87% and 90.54%,respectively.The accuracy rate,recall rate and F1 value of KdConv data set are 90.83%,87.66% and 89.21%,respectively,showing good results.The results show that this method can effectively solve the problems of insufficient data in the data set and mispronunciation of Chinese pinyin input,and expand the number of sentences and semantics of input sentences,thus improving the accuracy of inference method.(2)In view of the problem that the traditional neural network model for knowledge reasoning has low efficiency due to the inability of parallel computation,a Unified Transformer model is proposed to replace the traditional memory neural network model and add attention mechanism to complete knowledge reasoning.This method reduces the complexity of the model.It can also carry out parallel operation on the data,which improves the operation efficiency,solves the problem of long-term dependence in the model,and improves the inference accuracy.The experiment shows that the operation efficiency of the improved algorithm in this paper has been significantly improved,and in the experiment of DuConv data set,the coherence index has been improved by0.020,the information index by 0.115,the participation index by 0.049,and the humanization index by 0.059.In the experiment of KdConv data set,the coherence index increased by 0.100,the information index increased by 0.148,the participation index increased by 0.104,and the humanization index increased by 0.178.