Effects of automation failures during a visual inspection task on reliance and visual skill acquisition: A comparison between direct and indirect cueing

The present study investigated the use of two automated aids of different reliabilities in a luggage screening task. A Direct Cue consisting of a green circle around a potential target directs attention to a specific part of the luggage image, while an Indirect Cue, consisting of a green border around an image determined to have a target, does not. Direct Cues offer an advantage in visual inspection tasks because they guide attention to specific areas of the visual image and restricts search space. But this can also cause attentional tunneling effects. Furthermore, the reliance on automation may negatively impact manual performance after the aid is removed or is no longer available to the user. Thus, two issues were investigated in the current study: (1) how do failures in Direct and Indirect Cues affect user reliance, as reflected in performance, and (2) how does a complete failure affect performance after operators have had the use of an automated aid? Participants performed two sessions of the luggage screening task, first with the aid and then without. Control participants performed both sessions without the aid. Results suggest that reliance patterns were more optimal with the Direct Cue than with the Indirect Cue and performance with a more reliable Indirect Cue was not much better than a less reliable one but this was not true for the Direct Cue. The results also suggest that manual performance, when the aid was removed, was better for participants who had used the automated aid compared to control participants. The advantage of previously aided performance on subsequent manual performance was greatest for participants who had used the more reliable Direct Cue. Eye movement data also support these findings. Eye movement measures show that participants who used the Direct Cue spent more time inspecting the target compared to those who had used the Indirect Cue and this could have contributed to better recognition in the second session. These results are explained and the implications for the use of automated aids in visual inspection tasks are also discussed.