Research On Human-Robot Interaction Technology Based On Natural Language Object Retrieval

With the development of industrial automation and intelligence,intelligent robots have become more and more widely used in the manufacturing industry."Made in China 2025" clearly includes intelligent robot technology in the key scientific and technological fields to promote breakthroughs.The integration of robots,humans,and the environment is the hallmark of robot intelligence,and traditional interaction methods such as graphical interfaces and force feedback cannot meet the requirements of intelligence.Therefore,researching more intelligent human-robot interaction technology is an important way to realize robot intelligence,which can effectively improve the efficiency of human-machine collaboration and accelerate the process of manufacturing intelligence.Among many interaction methods,human language(natural language)is the most natural interface for human-robot interaction.Robots' understanding of human's natural language expressions can greatly improve the degree of intelligence of robots,so that robots and humans can interact like humans.To do this reliably,humans and robots must have a common understanding of the environment,which is the embodiment of the natural language object retrieval task: locate the target object in the global scene image based on the natural language representation.In this paper,natural language object retrieval is applied to human-robot interaction technology,so that the robot can locate the target object according to the description of the specific object in the scene image by the operator,and complete the task of grasping and placing the target object autonomously.Compared with interaction methods such as body gestures and graphical interfaces,human-robot interaction methods based on natural language object retrieval are more intelligent,which can effectively improve the efficiency of collaboration between operators and robots.The main work of this article is as follows:(1)A robot vision positioning system based on the Kinect camera and EPSON robot was built,and the Tsai two-step method was used for hand-eye calibration of the vision positioning system.Based on the camera imaging principle,the measurement method of the spatial position coordinates of the target object in the scene image is analyzed.Combined with the D-H parameters of the EPSON robot,the forward and inverse kinematics of the robot were solved,and the robot motion control algorithm was implemented using the C # programming language based on the results of the solution.(2)A candidate region extraction algorithm model based on the Faster-RCNN architecture is constructed.This algorithm is used to extract all candidate regions in the scene image and construct the candidate region set required by the natural language object retrieval algorithm.Collected and labeled the robot grabbing data set,and completed the training and performance evaluation of the candidate region extraction model on the data set.(3)Combining target detection technology and natural language processing technology,a natural language object retrieval algorithm based on ResNet and LSTM,a spatial context memory residual network(SCLR)model,is built.The model mainly includes four modules: feature extraction,word vector coding,LSTM module and representation prediction layer.The SCLR model was trained on the robot capture dataset using transfer learning methods,and the detection accuracy of the SCLR model was evaluated on the test set and the actual experimental scene.(4)A human-robot interaction experiment platform was set up,and the software design of the human-robot interaction platform was realized in a client-server mode.Completed the programming of SCLR model building,training,evaluation,and object retrieval process in the Linux server,and completed the programming of the candidate area extraction process and the robot control system process in the Windows client.And TCP / IP communication is used to achieve communication between processes and threads.Based on the human-robot interaction experiment platform,a speech-based human-robot interaction experiment was carried out,which verified the intelligence and efficiency of human-robot interaction technology based on natural language object retrieval.