The Implementation Of Spoken Language Understanding System Based On Pre-training Model

Spoken language understanding system enables users to interact with machines through spoken language.At present,the spoken language understanding system is widely used in banking,express delivery,insurance and other industries.It can meet the query needs in specific fields of users by understanding the questions and giving answers.In recent years,the research on spoken language understanding system that can automatically answer questions has become a new hot topic.From the traditional machine learning to the present realization based on deep learning method,the performance of spoken language understanding system has been continuously improved.However,the spoken language understanding system still has the problems of insufficient accuracy of intention recognition and slot filling.The reasons for the lack of accuracy are mainly reflected in three aspects: lack of data,unreasonable prediction of word slot filling sequence and insufficient use of domain characteristics.The contributions of this thesis are as follows:Firstly,this thesis builds a baseline model of intention recognition and slot filling.By studying the current research results of the spoken language understanding system,this thesis constructs the baseline model of intention recognition and slot filling.Then this thesis studies the components in the baseline model in order to determine the role and shortcomings of each component in the model and propose corresponding optimization methods.Secondly,this thesis constructs text-related pre-training model,conditional random field training model and domain feature training model to improve the baseline model.This thesis adds the text-related pre-training model to the baseline model in order to improve the generalization ability of the model,so that the trained model can accurately understand the text content of the out-of-set data.Then this thesis adds conditional random field model to the baseline model,which can learn the characteristics of the prediction sequence,so as to deal with the problem of unreasonable prediction of word slot filling sequence and improve the accuracy of word slot filling.Then this thesis constructs the domain feature model to further narrow the scope of intention recognition and improve its accuracy.Then this thesis constructs a comprehensive model including the above three optimization methods based on the baseline model.Finally,this thesis develops an spoken language system based on the comprehensive model.This thesis provides a software system that uses the WEB browser as an interactive platform.The system uses B/S architecture and Golang+My SQL technology for system development.This system completes automatic speech determination,problem analysis,answer acquisition,oral question answer,text question answer and other functions.This thesis combines theory with practice,completes the entire process of the spoken language system based on the comprehensive model,and reflects the practicability of the system through testing and using.