Research On Device Oriented Deep Learning Image Dialogue Help System

In recent years,due to the emergence of deep learning,the rapid development of artificial intelligence in various fields is obvious to all.As one of the hot research directions in the field of artificial intelligence,natural language understanding is also rising.Because natural language understanding combines deep learning,its system scheme and implementation have changed a lot.At present,dialogue devices emerge in endlessly in the market,but the application of dialogue system in solving the problem of using instruments and equipment is not much.In the near future,the research of image-based dialogue help system will be a good research direction.Compared with the paper-based help mode,the image dialogue help system is more convenient and faster to solve the problems that users encounter when using the device.This thesis is mainly for the use of equipment to help research.A device oriented deep learning image dialogue help system model is proposed,and the prototype system of the model is implemented.Finally,the model is applied in the field of instrumentation.Firstly,this thesis analyzes the development history and research status of the device oriented deep learning dialogue help system model at home and abroad,and briefly introduces the theoretical basis of the model,such as instance segmentation algorithm,word segmentation tool,multimodal fusion and concept dependency tree.Secondly,it analyzes many difficulties of language understanding in the dialogue help system of instruments and equipment with dialogue examples,and concludes that one of the reasons for the low accuracy of multimodal visual dialogue is that the natural language mode is often omitted in combination with the image,and the ambiguity caused by omission causes the model not to correctly understand the natural language text semantics Alignment and fusion of modal information.In view of this situation,this thesis proposes a device oriented deep learning image dialogue help system model.Before the traditional multi-modal information alignment and fusion,the ellipsis recovery disambiguation module based on the combination of image and knowledge is processed,and a series of ellipsis ambiguity problems are solved by using the knowledge of "only oscillograph can measure waveform".After ambiguity processing,the information that can establish the relationship between modes is more perfect than that without ambiguity processing.Thus,the help system can more accurately determine the main body of the instrument and equipment instance asked by the user.Then,based on the best language model and the lxmert multimodal model,the pre training model of the device oriented deep learning image dialogue help system is established.Before the pre training of the model,the relevant data set of the model is established,and the relevant solutions are proposed for the one to many relationship between the images and the problem in the data set.Then,the processing ideas and training results of the main modules in the model are analyzed with an example of bimodal input,and the recovery process of the ellipsis recovery module based on image and knowledge is emphatically analyzed.Finally,the deep learning dialogue help system model oriented to equipment is applied in the field of instrumentation.Taking the acquisition of specific equipment information as an example,the establishment process of knowledge base in the field of instrumentation is introduced in detail,and the acquired data is organized and managed in the form of instrument concept subordinate tree to the database.Then,it shows the application of the model in the field of instruments and meters with voice as the main part and interface as the auxiliary part.