Research On Mapping From Natural Language Instruction Paring To Target For Home Service Robot

With the continuous improvement of artificial intelligence and robotics,family service robots for daily life are becoming more and more popular.It is imperative to use natural language instructions to improve human-robot interaction experience.However,it is difficult for robots to recognize and analyze human natural language,so it is the most important part of human-computer interaction to analyze natural language instructions into the form that robots can handle to complete the mapping of targets.This topic will focus on this task and be detailed into the following aspects.First,natural language instruction parsing needs intention detection and slot filling.In order to take advantage of the correlation between the two,a joint model of natural language understanding based on graph LSTM and attention mechanism is proposed.Graph LSTM combines with attention mechanism to extract information from the original instruction and gets the result of intention detection by other operation.The intention detection parameters together with the preprocessed slot filling categories are used as slot filling inputs.Attention mechanism is used to extract information,and then processed by bidirectional recurrent neural network and conditional random field unit to output slot filling results.Secondly,in order to realize the mapping from instruction to target,an interactive image instruction fusion network is built.A cascaded hourglass network is used in the network to extract the information of the target object in the environment image,and the extracted parameters of the image are jointed with the preprocessed natural language instruction parameters to generate a target object position heat map and an uncertainty heat map.If the two heat maps are similar,the robot can determine the target object.If the two heat maps are not similar,it indicates that the target object or instruction is ambiguous,and the question answering network will be used to confirm the target.Then,a question answering system is designed when the target object or instruction is ambiguous.The robot can ask the user for supplementary information,and the result will be added into the original instruction.The image instruction fusion network reanalyzes the environment image and the new instruction,and finally obtains the confidence heat map of the target object to complete the mapping from instruction to target.Finally,the effectiveness of the proposed algorithm and the feasibility of the system are verified by experiments,and other related experiments are compared,and the experimental results of the proposed algorithm are analyzed.