Research On Industrial Design Evaluation System Of Multi-modal Human-Computer Interaction In MR Environment

At present,the evaluation model of products in industrial design is a physical prototype,a digital prototype or a virtual prototype.The main problems of physical prototypes are high cost,long cycle and fidelity.The main problem of digital prototype is that it depends on the application software,which has high requirements on equipment and users.The virtual prototype can realize real-time interaction and overcome the above two methods.The main reason for the interaction is based on the traditional graphical interface of the computer and the mouse and keyboard.It is not a natural interaction method.At the same time,both the digital prototype and the virtual prototype require the user to operate on the computer,which is not convenient enough to limit the naturalness of user interaction.In recent years,with the development of multimedia software and hardware technology,the research on human-computer interaction has been extended to the field of Mixed Reality(MR).Holo Lens is a head-mounted MR device developed by Microsoft Corporation.It emerged as a huge technological innovation in the field of mixed reality.It uses a special holographic processing unit to generate holographic images in real-world environments,helping users to pass gestures,voice,vision,etc.A variety of channels for the most natural way of operation.The development and design of industrial design evaluation system for multi-modal human-computer interaction in MR environment provides a new design and evaluation method for industrial design.The main work and contributions of this paper include:Firstly,the interaction mode and interaction interface of multi-modal human-computer interaction system in MR environment are analyzed.Based on these differences,a multi-modal human-computer interaction system framework based on three-layer architecture and multi-modality based on this framework are proposed.Human-computer interaction process.Secondly,the core technologies of holographic display technology,three-dimensional registration technology and human-computer interactiontechnology in MR platform are studied.According to human body characteristics and MR equipment characteristics,the reference factors of multimedia-based human-computer interaction design based on MR are summarized from the perspective of ergonomics.Then,through the analysis of system requirements and problem domains,the structure and function of the industrial design evaluation system for multi-modal human-computer interaction in MR environment are proposed.Aiming at the complex modeling and large amount of data in industrial design,a model optimization method based on Physical Rendering(PBR)is proposed.Finally,taking the cleaning car design as an example,the multi-modal human-computer interaction cleaning vehicle design evaluation system was completed by means of Holo Lens,and the digital virtual cleaning vehicle model and attributes can be real-time,dynamic and seamless through interactive dialogue with people.Superimposed in the actual use scene,realizing the realistic reproduction of the cleaning car model in the real environment and the user's evaluation.The superiority of the system was evaluated through comparative experiments.