Intention Understanding And Initiative Service Strategies For Human-robot Interaction

With the development of human-robot interaction,emotion computing,and other theories and technologies,service robots have been integrated into all fields of people’s lives.However,for human-robot interaction mode,the passive service mode that relies on instruction recognition can no longer meet people’s demands.If robots can analyse users’ emotions and scene information to infer users’ intentions,and make autonomous decisions to provide services,it will have important research significance in the field of human-robot interaction technology,and will also help service robots better integrate into practical applications scenes.At present,the related work has problems such as mechanization of service mode in human-robot interaction,insufficient analysis of relevant information by intention understanding methods,insufficient labeled data in actual scenarios,and different abilities of multimodal emotional features to represent different emotions,making it difficult to meet the needs of initiative service implementation in real scenarios.Therefore,in this paper,the research on the initiative service model is carried out.The information related to the user’s intention is analyzed from the actual needs,and the multi-channel feature fusion,multi-modal emotion recognition method under small sample data,and the intention understanding method in the driver scene are studied.Through these studies,the robot can mine and understand deep cognitive information of users in the real scene,provide a theoretical basis for related fields,and explore the further development direction of human-robot interaction mode.The main research results and innovations are as follows.(1)An initiative service model driven by deep cognitive information for service robot is proposedAiming at the problems of mechanization and passivation of the current service robot service model,an initiative service model driven by deep cognitive information is proposed,and an exploratory research on initiative service is initiated.Different from the previous human-robot interaction model based on instruction recognition,the initiative service model established in this paper focuses on mining the deep cognitive information(including emotions and intentions)of users in human-robot interaction,based on this,autonomous reasoning decisions are made to realize services.Taking the driving scene as an application example,the layers in the initiative service model are explained in detail,which lays an important foundation for the research in the following chapters.(2)An audio-visual emotion recognition method based on multi-channel feature fusion is proposedTo solve the problem that emotional features have different recognition performance for different types of emotion,an audio-visual emotion recognition method based on multi-channel feature fusion is proposed.To capture the advantages of different kinds of features for emotional representation,this paper extracts artificial features and deep features in different modalities.Considering the difference between two types of features,a multi-class SVM recognition sub-model is established for the artificial features,and a deep learning recognition sub-model is established for the deep features,the four sub-models are fused by the ensemble learning method.To improve the sensitivity of the model to user’s emotions,a multi-task learning mechanism is introduced into the deep learning sub-recognition model,and gender recognition is selected as an auxiliary task to establish a multi-task convolutional neural network model.The experimental results show that the emotion recognition method proposed in this chapter can achieve higher recognition accuracy.Experimental results show that the proposed method obtained higher recognition accuracy.(3)An emotion recognition method based on dual visual modes for natural driving environment is proposedDue to the insufficient labeled emotional data in the driving scene,single modal data is vulnerable to driving tasks and illumination,and the equipment limitations and high real-time requirements in the driving scene,an emotion recognition method based on dual visual modes for natural driving environment is proposed to monitor the driver’s emotional state during driving.Taking into account the real-time performance and accuracy of emotion recognition in the driver scene,the lightweight network model Mobile Net is adopted to establish driver emotion recognition model.A transfer learning mechanism is introduced to Mobile Net model,in which a large sample data set related to emotion recognition task is selected to pre-train the model,and then use the driver data to adjust the model to improve the generalization of the model in driving scene through parameter transfer.In addition,considering that facial expression data is easily disturbed in driving,this paper analyzes the relationship between driver’s body posture information and driver’s emotion for the first time,and uses body posture information as supplementary information to perform dual-modal emotion recognition.The experimental results show that the driver emotion recognition method proposed in this chapter achieves higher recognition accuracy,and the fusion of body posture and facial expression has a better recognition effect than facial expression emotion recognition alone.(4)A driver intention understanding method based on context analysis is proposed.The prediction of driver’s intention can reduce occurrence of traffic accidents.Extant intention understanding methods mostly take action information or environmental information in specific scenes as input,however,the driving process is affected by many factors,such as driving environment and drivers.The single information can not accurately predict driver’s driving intention.To solve this problem,a driver intention understanding method based on scene related information analysis is proposed.By simulating human thinking in the process of reasoning and judging comprehensively considering a variety of information,this paper simultaneously analyzes the influence of users’ personal information and scene information on intention,and improves the accuracy of intention reasoning.To make the description of various information consistent in intention reasoning process,the scene information is characterized based on driver’s understanding.To ensure the consistency of evaluation,fuzzy analytic hierarchy process is used to calculate the influence degree of feature state on different intentions,and final driving intentions of drivers are calculated by fuzzy Bayesian method.It is verified that the proposed method can accurately infer driver’s intention and provide drivers with effective service strategies through experiments in different driving scenarios.The initiative service model for service robots is established and the methods of emotion recognition and intention understanding is studied in this paper,which provides a theoretical basis for the robot to understand user’s emotion,intention and demands,and provides a practical solution for realizing the intelligence and humanization of human-robot interaction,so that it can perceive human’s emotion through audio-visual and provide initiatively servive through independent decision-making.