A Comparison of Graphics-Based versus Text-Based Online Probe Methods for Predicting Performance of Air Traffic Controller

The present study compared the validity of a new graphic probe presentation technique for the Situation Awareness Present Method (SPAM) with the established traditional text-based probe presentation method. Three primary research questions were to be answered: 1) Which method of assessing situation awareness will best predict air traffic controller (ATCo) performance? 2) Will the method of probe administration, graphics-based vs. text-based, affect the time to respond to the question?, and 3) Will the method of administration, graphics-based vs. text-based, affect the accuracy of probe responses? Participants performed a simulated air traffic control task using a Multi Aircraft Control System (MACS), which is a simulated mid-fidelity ATCo environment. SPAM probe questions were presented on an adjacent computer. Two manipulations were used during the course of this study: 1) the amount of aircraft in the sector at a given time (traffic density), and 2) the probe presentation technique (Graphic vs. Text-based), with the traditional SPAM dependent variables of probe accuracy, subjective workload, workload latency, and probe question latency. First, the findings of the present study show the shorter response time of text-based probes. Second, the findings of the present study show text-based probes to generate more accurate participant responses. Lastly, the findings of the present study suggest both text-based and graphic probes predict aspects of ATCo performance. However, only specific text-based probe questions predict Losses of Separation, the primary measurement of ATCo safety. While significant findings were discovered during the course of the present experiment, further research is needed to determine the validity and reliability of the present findings.