| This study investigated the measurement of users' subjective satisfaction in usability testing. Three different methods of administering satisfaction measures were compared to determine whether priming techniques are capable of influencing users' satisfaction ratings. Additionally, users' rationales for their satisfaction ratings were examined to help researchers understand why users sometimes rate systems in a manner that is inconsistent with their observed performances. Finally, the relationships between the primary attributes of usability—efficiency, effectiveness, and satisfaction—and users' rationales for their satisfaction were investigated. Forty-five undergraduate education students from a large mid-western university were randomly assigned to the three test conditions and participated in 90-minute usability sessions using a web-based enterprise portal system.; No significant difference was found between each group's post-test satisfaction ratings, suggesting that priming has no discernible impact on users' actual rating behaviors. Many users did perceive benefits from priming, however. These included an increased awareness of the expectations for the test prior to beginning the usability session, and assistance with remembering their experience throughout the test as a result of their post-task satisfaction ratings. Users expressed thirteen common rationales for their satisfaction ratings, including Utility and usefulness are key; Just give me some time; Clear and helpful; It's my fault, not the system's; Time equals satisfaction; Everything in its place; The paradox of information quantity; I can see its potential; Feedback is important; If someone would just tell me; Confusing terminology; Getting it right…finally; and I wouldn't do that task anyway. Significant negative correlations were discovered between users' satisfaction and time spent per task, and between time spent per task and users' effectiveness. A significant positive correlation was discovered between users' satisfaction and users' effectiveness. The findings from this study should help usability researchers better understand and interpret their efforts at measuring users' satisfaction. Additionally, they inform the developers of web-based portals of some of the critical design attributes required for their users' satisfaction. |
