Three-dimensional Multimodal User Interface Paradigm And Usability Evaluation Method

Three-dimensional multimodal interaction is a way of human-computer interaction(HCI)that is closer to the real world cognitive.It supports a variety of input and output devices,which can meet the interaction needs of different user groups.It has been widely used in medical,education and many other fields.Through the threedimensional multimodal user interface,people can perform the operation task more easily and perceive the system state with a stronger sense of immersion and realistic in the process of HCI.Therefore,the research on user-centered three-dimensional multimodal user interface design has important practical value and significance.There are two urgent problems to be solved: three-dimensional multimodal user interface lacks a unified interface paradigm,which brings difficulties to interface development;the complexity of the multimodal interaction information sources leads to the lack of useful usability evaluation index system and quantitative usability evaluation method.Aiming at the above two problems,this paper carries out research based on the three-dimensional multimodal interaction platform of the project team,including three aspects: based on the Norman's model,the user interaction behavior is analyzed,and the reality-based interaction elements are introduced to establish a three-dimensional multimodal user behavior model and a real-world mixed control metaphor framework;based on the three-dimensional multimodal user behavior model and metaphor framework,a three-dimensional multimodal user interface paradigm is proposed and applied to interface design;according to the characteristics of three-dimensional multimodal interaction,the usability evaluation index system is established,and the usability evaluation method is proposed.The main work and innovation of this paper are as follows:(1)According to the characteristics of user interaction behavior and threedimensional multimodal user interface,the three-dimensional multimodal user behavior model and the interface metaphor framework are established respectively.In order to solve the problem of insufficient analysis of interaction process in the existing user behavior model,a three-dimensional multimodal user behavior model is established based on the Norman's model.The model established accurately analyzes the user behavior process in the three-dimensional multimodal environment,describes the influence and action mode of reality-based interaction elements on the interaction process and provides guidelines for the design of three-dimensional multimodal user interface.In order to solve the matching problem between the interaction domain and the digital application domain,this paper classifies the interface elements,constructs the conceptual model of three-dimensional multimodal interface metaphor,and determines the mapping relationship between metaphor and interface elements.Based on this mapping relationship,a real-world mixed control metaphor framework is established through operation method,pragmatics method and structure method,which serves as the metaphor basis of interface design to make the three-dimensional multimodal interface work based on the user's existing knowledge.(2)Aiming at the three-dimensional multimodal user behavior model and metaphor framework,a three-dimensional multimodal user interface paradigm3 DSOEM is proposed,and a three-dimensional multimodal globe teaching software interface is designed.Referring to the division of interface components of WIMP paradigm,based on the three-dimensional multimodal user behavior model and interface metaphor framework,the necessary components of three-dimensional multimodal user interface are analyzed,and the three-dimensional multimodal user interface paradigm 3DSOEM is proposed.The paradigm narrows the interaction gulfs by considering human factors and realistic factors.In order to verify the feasibility of3 DSOEM paradigm,a three-dimensional multimodal globe teaching software user interface is designed.The experimental results show that the interface has higher interaction efficiency and lower error rate compared with WIMP interface with the same function,and brings more comfortable interaction experience to users.(3)Aiming at the design requirements and influencing factors of three-dimensional multimodal user interface usability,the usability evaluation index system is established,and the usability evaluation method of three-dimensional multimodal user interface is proposed.This paper analyzes the evaluation index,influencing factors and evaluation methods of user interface from three aspects of system performance,task performance and user subjectivity,and establishes a three-dimensional multimodal user interface usability evaluation index system.For the weight influence of specific indexes involved in the index system on the overall usability,a usability weight model is established,which is based on analytic hierarchy process.The overall usability is divided according to the evaluation index system,and the interactive data is standardized through the evaluation matrix to obtain the weight value of each evaluation index.The usability evaluation method provides an efficient usability evaluation method for different threedimensional multimodal user interfaces.