Design and Evaluation of Auditory-Supported Air Gesture Controls in Vehicle

The number of visual distraction-caused crashes highlights a need for non-visual information displays in vehicles. Auditory-supported air gesture controls could fill that need. This dissertation covers four experiments that aim to explore the design auditory-supported air gesture system and examine its real-world influence on driving performance. The first three experiments compared different prototype gesture control designs as participants used the systems in a driving simulator. The fourth experiment sought to answer more basic questions about how auditory displays influence performance in target acquisition tasks. Results from experiment 1 offered optimism for the potential of auditory-supported displays for navigating simple menus by showing a decrease in off-road glance time compared to visual-only displays. Experiment 1 also showed a need to keep menu items small in number but large in size. Results from experiment 2 showed auditory-supported air gesture controls can result in safer driving performance relative to touchscreens, but at the cost of slight decrements in menu task performance. Results from experiment 3 showed that drivers can navigate through simple menu structures totally eyes-free, with no visual displays, even with less effort compared to visual displays and visual plus auditory displays. Experiment 4 showed that auditory displays convey information and allow for accurate target selection, but result in slower selections and relatively less accurate selections compared to displays with visual information, especially for more difficult target selections. Overall, the experimental data highlight potential for auditory-supported air gesture controls for increasing eyes-on-road time relative to visual displays both in touchscreens and air gesture controls. However, this benefit came at a slight cost to target selection performance as participants generally took longer to process auditory information in simple target acquisition tasks. Experimental results are discussed in the context of multiple resource theory and Fitts's law. Design guidelines and future work are also discussed.