Research On User Attention Computing Method In Human-robot Natural Interaction Scenario

With the development of artificial intelligence,human-robot natural interaction(HRNI)has gradually entered people's field of vision,which marks that the research of human-robot interaction has entered a new stage.Human-robot natural interaction refers to the process of information exchange between human and robot through natural language,action,expression and other different ways of expression.Attention computing refers to the quantitative calculation of human's attention concentration in human-robot natural interaction by using robot's perception data.Accurate perception of user's interaction attention is the basis of evaluating user's interaction intention,providing service actively and controlling interaction process adaptively.In the process of natural interaction in the absence of user attention calculation,it is easy to appear invalid interaction such as self talk,which reduces the degree of robot intelligence.In recent years,most of the robots come out to open the interaction through wake-up words,touch or face detection.The way of wake-up words and touch belongs to passive wake-up with low degree of intelligence;the way of face detection belongs to active wake-up,but it is easy to wake up by mistake.Therefore,attention computing is of great significance to improve the intelligent level of robot,and it is one of the important technologies of human-robot natural interaction.In this thesis,by analyzing the current research work of human-robot natural interaction,taking the calculation of user attention in human-robot natural interaction as the research object,two methods of user attention calculation in human-robot natural interaction scenario are proposed:1)Aiming at the problem of static feature extraction and modeling of user attention in human-robot interaction,a multi-feature fusion method for static attention calculation of user is proposed.In this thesis,RGB and RGBD images of users are acquired by depth camera.Six different features,including face information,head posture,lip distance,interaction distance,human body azimuth and human body deflection angle,are extracted from each image.The random forest model is used to fuse these six different features to establish user attention regression model.In order to verify the effectiveness of the proposed method,this thesis constructs a static HCI attention dataset.The experimental results show that the method can effectively calculate users' static attention.2)To tackle the instability of attention modeling in human-robot interaction under the condition of changeable dynamic characteristics of users,this thesis proposes a dynamic attention calculation method of user based on attention LSTM network.In this thesis,a depth camera is used to obtain RGB and RGBD video of the user,and six temporal and sequential feature models of face information,head posture,lip distance,interaction distance,human body azimuth and human body deflection angle are constructed for consecutive frame images.The attention-LSTM network is used to establish the user attention calculation model.In order to verify the effectiveness of the proposed method,this thesis constructs a dynamic attention score data set with six attention characteristics.The experimental results show that compared with the static attention calculation method,this method can effectively analyze the user's attention persistence in the dynamic environment.3)Based on the two attention computing methods proposed in this thesis,an attention computing system is constructed through Py Qt5 application framework.The application of the system in robot active wake-up and interactive process control is introduced in detail.