Research On Multi-modal Human-computer Interaction Design In Cockpit

The cognition of human is supposed to work in a multimodal parallel manner and overload usage of a single channel could easily cause an imbalance allocation of concentration, exacerbating cognitive resources consumption. Aviators’perception to the flying status and environment is obtained mainly through visual channel. Facing a complicated and changing information presentation in cockpit, the bandwidth of visual channel will be in a high working load state for a long time, thus can easily leading to an illusion and delay in the visual channel. By introducing multichannel modal HCI techniques, pilots’ cognitive behavior through auditory and haptic perception, supplementary to visual one, could be aroused, in which way it could reduce visual working load, improve the efficiency and accuracy of info conception.This thesis focuses on modern airliners, summaries the procedure of flying routines, conducts a cognitive behavioral analysis taking account to HCI scenarios, subject to Cognitive Resources Theory, and it also categorizes flight mission based on Petri net model. It studies the characteristics and the method of applications of those factors for visual, auditory and haptic channel, respectively. This topic also clarifies the limitations and interactions of each single channel. It summarizes different combination modes of those factors from visual, auditory and haptic channel and their application environment, limitations. It builds an improvement principle for HCI design according to the features of flight mission. This thesis proposes theoretical schemes and analysis specific to four typical flight missions, and conducts empirical assessment on these improvement schemes with the help of E-prime and then verifies the feasibility and effectiveness of multimodal HCI design of a flight deck.