The distributed cognitive walkthrough: The impact of differences in cognitive theory on usability evaluation

A primary goal of usability related fields is the creation of products and services that conform to or leverage theoretical views of human cognition. Therefore, most usability related methodologies are directly based on theories of human cognition. Existing user-centered design (UCD) methods are based on cognitivist theoretical views, which hold that the locus of cognition is the brain. Opposing post-cognitivist views within the cognitive science field, such as distributed and extended cognition are not represented in current UCD methods. Our research represents a first step towards understanding the impact of differences between these opposing theories of cognition on usability evaluation and design. In examining the differential impact of opposing cognitive views (cognitivist versus post-cognitivist), three main research questions were explored using two experimental studies.;The first experimental study explored the impact of walkthrough evaluation methods with a focus on an individual user versus a focus on interaction between internal and external resources, and the number of users in a scenario on the actionability, usability relevance, and reported severity of usability evaluations. Usability evaluations were conducted by 42 participants using either the Cognitive Walkthrough (representing cognitivist theoretical views) or the Distributed Cognitive Walkthrough (novel method developed as part of this research, representing the distributed and extended cognitive views) with scenarios set in a coffee shop. Use of two-way ANOVA showed a significant increase in all three dependent measures (actionability, usability relevance, and reported severity), with use of the Distributed Cognitive Walkthrough method over the Cognitive Walkthrough method.;A second experimental study explored the impact of materiality of artifacts (paper cup versus digital interface) on the actionability, usability relevance, and reported severity of usability evaluations. Usability evaluations were conducted by 22 participants using the Distributed Cognitive Walkthrough with scenarios set in a coffee shop that focused on either a paper coffee cup or a digital order tracking system interface. Use of ANOVA showed a significant increase in reported severity for scenarios with the digital interface over scenarios with the paper cup. While differences in actionability and usability relevance by materiality were non-significant, interesting aspects of the differences are discussed.;Beyond the individual results stated above, our findings suggest that an increased focus on the interaction of internal and external cognitive resources and variation in materiality of these resources during evaluation can result in making useful, yet invisible resources more visible, with respect to how an activity is being accomplished. This aspect of addressing the paradoxical invisibility of useful and usable artifacts is a key contribution of this research, showing how even simple differences in evaluation and design methods can aid in making visible these complex interactions between internal and external resources for cognition.