Research And Implementation Of Multimodal Dialogue Management System

Currently, the robots shift gradually from industrial robots to service robots. One important characteristic of service robots is that they must dialogue with human through natural language. In human-robot dialogue system, dialogue management is the indispensable kernel of the system. Traditional dialogue management mainly makes use of language. Nevertheless, in the service robot human communication process, not only the language information, but also the dialogue scene information, such as interlocutors’ expressions, gestures, as well as the visual scene information talked about. This information may make a great influence with dialogue act. Therefore, it is very necessary to conduct dialogue management based on multimodal information of visual and language.This article conducted the research of multimodal dialogue management based on the existing dialogue management technology. Firstly, this paper raised a dialogue management model of intention analysis based. This model mainly conducted a hierarchical analysis about the user dialogue’s intention, and then designed a dialogue management method combining a finite state machines and a task tree. To verify the hierarchical intention analysis model proposed by this paper, we conducted a series of classification experiment. According to the experiment result, the hierarchical intention analysis model is effective.Finally, this paper synthesized all related technologies of dialogue system, applied the model of multimodal dialogue management proposed before, and designed to realize one multimodal dialogue system based on NAO robot. This system simulated the children’s cognitive process on things’ language and image, and it made the robot learn things’ names and image feathers in the scene, such as color and shape. What’s more, user may query the knowledge the robot have learned, and the system also can give the appropriate reply after retrieving its own knowledge base. The experimental results show that the dialogue management model based on intention analysis proposed by this paper can effectively ensure the fluency and naturalness of the multimodal human-robot dialogue.