Information Perception And Feedback Mechanism And Key Techniques Of Multi-modality Human-robot Interaction For Service Robots

With the increase of the elderly population and the growing health care cost, the role of service robot in aiding the disabled and elderly is becoming important, and many researchers in the world have paid much attention to the healthcare robot and rehabilitation robot. To get natural and harmonious communication between user and service robot, the information perception and feedback ability, and interaction ability for service robots become more important in many key issues, but currently these capabilities of service robots are in preliminary level, therefore, there is much work to be done to improve the performance of service robots.Considering the characteristics of service robots to interact with the elderly and disabled, we systematically researched the information perception and feedback mechanism, and multi-modality human-robot interaction approaches, and then set up a multi-modality and human-robot natural interaction system which has multiple information perception and feedback functions.In the dissertation, we firstly had a research on some natural response modalities, such as eye-gaze input, speech recognition and syntheses, and mechanical touch modality, and presented the main-sub modality collaborative work mode for these response modalities, by this collaborative method, the work load of single modality has been decreased, and the work efficiency and interaction robust for these modalities have been improved. In order to improve the adaptation of eye-gaze input modality to environment illumination, a set of image capturing equipment and image binary threshold model have been put forward, and some strategies and control methods have been presented to decrease the influences on the system caused by users head movement, by these means, the adaptation of the system to environment illumination and users head movement have been improved.To get natural and harmonious interaction, a kind of human-robot interaction architecture based on agent has been proposed, and the user model and interaction protocol in this architecture have been presented. According to the format of the interaction protocol, a set of task process management method has been put up in the light of task lifecycle, and then at the semantic decision–level, the frame-based information fusion approaches has been applied to integrate the interactive semantics from different interaction modalities. Some work on user telepresence in the"Human-Interface-Machine(Robot)"architecture, has been carried out, and a kind of information perception and feedback method by multiple modes has been presented, all these have increased the naturalness for user to get feedback information and interact with service robots.Finally, in the"Human-Interface-Machine(Robot)"system formed by service robot and user, some experiments about interface have been done with the cognitive psychology test method, by this way, a kind of adaptive interface catering to the elderly and disabled users personal characteristics can be gotten, and some other experiments have been carried out with ergonomics test method to get the working parameters range of some response modalities, moreover, this method has been used...