Research On Multimodal Interaction Design Based On In-vehicle Information System Tasks

With the continuous development of vehicle communication technology,the invehicle information system,as the endpoint of the intelligent transportation system,is evolving into being more multi-functional and intelligent with constant network connection and has been widely adopted as a common vehicle configuration.Covering music playing,navigation,gaming and many other services,the in-vehicle information system has brought immersive entertainment experience and convenience to drivers.In the meantime,however,the increased frequency of in-vehicle information system usage could divert driver’s attention and occupy the resources that should have been used for the main driving task(i.e.processing on-road information vs.processing information from in-vehicle systems),leading to drivers being distracted and driving safety being negatively impacted.Therefore,this paper attempts to introduce multimodal interaction into in-vehicle information system tasks,and reduce the driving danger brought by in-vehicle information system and improve the user experience by rationally distributing the driver’s attention resources.This research focuses on four aspects: theoretical research,experimental research,construction of HMI design and evaluation model,and design practice for in-vehicle information system.Firstly,this study points out that in a complex human-computer interaction system,driver’s multitasking could lead to conflicts in resources to fulfill each task whereas introducing multimodal interaction could improve driving safety and user satisfaction.Based on researching driving safety and multimodal human-computer interaction usability,a multimodal interaction design framework for operating invehicle information system is constructed.Secondly,Card classification method is used to aggregate and classify more than 300 manipulation tasks to fulfill multiple in-vehicle system functions into four basic task categories: entry,selection,activation/ deactivation,adjustment.For each of the four task categories,a set of different interaction modes is developed via literature research,and subjects are asked to follow those interaction modes sequentially in the scenario of playing certain music while driving,where three indicators(driving load,driver distraction,and driver satisfaction)are used to evaluate how each interaction method would occupy driver’s resources and affect driving.Thirdly,based on above findings a multimodal human-computer interaction evaluation model for in-vehicle information system operation is constructed,and the design principle and design flow of human interaction with in-vehicle information system are proposed.Finally,through design practice,the applicability and feasibility of the theoretical framework and design model constructed in this paper are verified.The significance of this topic lies in addressing open questions of multimodal interaction design for in-vehicle information system operation.It analyzes usability and potential influences on driving safety of various interaction modes under the four task categories,and constructs the multimodal human-computer interaction design and evaluation model which effectively provides guidance for future design.