Research On Intent Recognition And Semantic Slot Extraction Algorithms Based On BERT

The human-machine dialogue system includes three key modules:natural language understanding,dialogue management,and natural language generation.Among them,the natural language understanding module includes two major tasks:intent recognition and semantic slot extraction.In 2018,Google proposed a pre-trained language model BERT based on two-way Transformer,which achieved the most advanced results in many natural language processing downstream tasks.Therefore,this paper mainly studies how to improve the recognition accuracy of natural language understanding modules based on BERT.Based on this,the following work has been done:(1)Proposing a BERT-Bi LSTM-CRF semantic slot extraction algorithm based on BERT.Compared with the traditional Bi LSTM-CRF information extraction model,the BERT-based semantic slot extraction model has stronger semantic encoding capabilities and better generalization performance for small data sets.Semantic slot extraction experiments were performed on the ATIS and SNIPS datasets respectively.Because the bidirectional Transformer feature extractor used by BERT has stronger semantic encoding capabilities,and because of its transfer learning characteristics,it is more suitable for small samples and other advantages.Bi LSTM,Bi LSTM-CRF,and other semantic slot extraction models.BERT shows higher recall and accuracy.The_#Score on the ATIS and SNIPS datasets have reached 95.83 and 92.40,respectively.For the Bi LSTM-CRF model,the_#Score has been increased by 2.16 and 1.97.(2)Training the intent recognition and semantic slot extraction tasks in the traditional model separately,ignoring the correlation between the intent category and the semantic slot,and establishing a unified joint loss function by minimizing the loss of the two subtasks to achieve multi-tasking Fusion learning.In order to further strengthen the fusion of the two in multi-task learning,based on the addition of the loss function,a model based on the attention mechanism between intent and semantic slot is proposed,and the single-task,multi-task and multi-task are compared respectively.The performance of three models of attention mechanism is introduced below.The experimental results show that the multitasking fusion training has improved the_#Score on the ATIS and SNIPS datasets,which are 96.12 and93.84,which are 0.29 and 1.44.After the attention mechanism is introduced,the_#Score is improved by 0.58 and 1.86 compared to the simple multi-task training method,which is significantly greater than the improvement of the single task to multi-task training,which shows that the attention mechanism has a more obvious effect on the model.,Can more effectively improve the accuracy of model intent recognition and semantic slot extraction tasks.(3)Based on the study of BERT-based natural language understanding algorithms,this paper designs and implements a human-computer dialogue system for flight booking scenarios,supporting multi-round conversation functions such as active questioning and information confirmation.Detailed functional tests have been made from the system interface and API interface respectively.The system can run normally and stably,and complete the task of booking flights for users.