Research On Spoken Language Understanding Model Based On Non-slot Information And Memory Position Encoding

The wave of artificial intelligence is driving the development of dialog system technology.For an intelligent dialog system,understanding the semantic information of the user's speech has naturally become the first priority.However,natural language has great flexibility and ambiguity.In order to get the semantic information,we must tap the semantic features.This paper first studies the semantic understanding of singleturn dialogues and observes that non-slot information has an important impact on the semantic understanding of single-turn dialogues.Further,when the number of turns of dialogue increases,non-slot information is scattered in the context of the dialogue context.The semantic understanding of multi-turn dialogues needs to extract useful information from the historical context information,this paper studies the problem of semantic understanding in multiple turns of scenarios and proposes a memory position encoding and memory gate mechanism to solve the problem of the use of historical dialogue information.The specific work is as follows:First,this paper studies the problem of semantic understanding of single-turn dialogues,and proposes a Multi-Task Non-Slot Attention(MTNSA)model based on the impact of non-slot information characteristics on semantic understanding.The MTNSA model employs an external pre-trained model called BERT as the encoder of semantic information.By decoding non-slot information,it uses self-attention to strengthen non-slot information to improve semantic understanding.As a multi-task joint model,MTNSA can simultaneously perform three tasks of domain classification,intent recognition,and slot filling.Finally,it is experimentally verified that the reinforcement of non-slot information is helpful for semantic understanding.Secondly,according to the constraints between domains,intents,and slots,a mask method is proposed.The explicit constraint relationships between domains,intents,and slots are statistically represented as one-hot mask vectors.The model outputs multiple probability distribution vectors and uses mask vectors to exclude irrelevant terms,and then takes argmax to obtain labels.The add-on experiments with multiple classic methods have verified that the Mask method can improve the sentence accuracy rate.Further,when the number of dialogue turns increases,the non-slot information that affects the user's current sentence recognition is scattered in the context of the historical dialogue.In this paper,memory network-based methods are applied to encode and store historical information,and to improve the problem of memory sequence information loss in traditional memory networks,and to introduce memory position encoding to the outside world.In memory storage,the positional encoding is used to strengthen the logic of the memory,to give different weights to the historical memory of different locations,and to use the attention mechanism to obtain the context information vector by weighted summing from the position-enhanced memory,and by setting the memory.The gate mechanism filters the noise data in the information vector below.Experiments show that the introduction of position coding and memory gate mechanism is effective for context semantic understanding.