An Accessible Cognitive Modeling Tool for Evaluation of Human-Automation Interaction in the Systems Design Process

One of the main limitations of existing approaches to complex human-in-the-loop system design is the requirement for empirical data as a basis for alternative design selections. Experimental studies can be time consuming and costly. In addition, design decisions are often based on collections of design guidelines with limited theoretical explanations for why such guidelines may be effective from a human information processing (HIP) perspective. The lack of a cognitive explanation limits understanding of when and how guidelines can be applied. In order to better support conceptual design, various cognitive modeling techniques and tools have been developed based on HIP architectures. However, these techniques and tools also have several limitations from a design perspective. Existing tools are not easy to use and designers or developers may need extensive training and practice in use. Furthermore, there is currently no fundamental set of tool capabilities, such as providing a task workload analysis or identifying patterns of HIP (e.g., memory use), simulating visual object use (e.g., eye movements), providing interface design support, etc. This research integrated various capabilities of existing modeling tools into a new enhanced cognitive modeling language based on GOMS (Goals, Operators, Methods, and Selection Rules).;While GOMS modeling methods and GOMS language are considered easy to learn and use, the modeling approach has several limitations. The language is limited to representing expert behavior in tasks. In addition, GOMS models do not support modeling of lower-level behaviors, such as specific forms of visual processing (e.g., foveal vs. peripheral) as well as parallel processing of visual and motor operations. Another major limitation of GOMS modeling is that the operator time estimates are deterministic. Therefore, model output may not accurately represent individual differences in performance or the stochastic nature of human behavior in complex tasks. On the basis of these limitations, there is a need to develop a new cognitive modeling tools. This research developed a computational cognitive modeling tool using the enhanced-GOMS language to aid complex system designers in assessing the potential for automation-induced human performance errors. Application of the tool focused on pilot use of automation on the commercial aircraft flight-deck. The GOMS language was extended for application to this context. Output of the tool was compared with experiment data for validation purposes. It is expected that this approach would allow for accurate explanation and prediction of user behaviors during the design of complex aircraft systems and/or interfaces.;The modeling tool development included: a prototyping module; a user activity flow diagram (AFD) development module; an AFD to E-GOMS language translator; an E-GOMSL editor; a model parser and compiler; and a model simulation tool and report generator. This research used Microsoft (MS) Excel with Visual Basic for Application (VBA) Macros in the modeling tool development. A designer is able to use images to define a prototype including visual and non-visual objects (e.g., auditory interfaces). The designer can also develop an AFD based on the results of a cognitive task analysis (CTA) involving expert operators. The AFD is directly translated to E-GOMS by the translator module. Alternatively, the designer is able to code an E-GOMS code model using the E-GOMS editor. After coding the model, the parser and compiler can be used to obtain a quantitative analysis including task execution times based on stochastic estimates of individual operation times and a workload analysis. With these results and the GOMSL models, the simulator can be used to visualize the flow of HIP, represent patterns in HIP, and present a graphical workload analysis. Last, the report generator can be used to produce a summary of the quantitative analysis and simulation.;In order to validate the results of the modeling tool, a flight simulator experiment was conducted with a futuristic forms of cockpit automation (a Continuous Descent Approach (CDA) tool for flight route replanning). A CTA was conducted to identify pilot behaviors and to generate a data set for validation of the cognitive model output. An E-GOMS model of pilot behavior with the CDA tool was compared against the experiment data. Results demonstrated that the modeling approach provided as it was expected that this approach would allow for accurate explanation and prediction of pilot cockpit behaviors and that the tool would be useful during the design of complex aircraft systems and/or interfaces.