Abstract

Verbal protocols are the primary tool for understanding users' task-solving behaviours during usability testing. We describe a qualitative study that examined the utility of combining a concurrent and retrospective think-aloud within the same usability test. The results indicate that while there was significant overlap between the types of utterances produced during each think-aloud, the retrospective phase produced more verbalizations that were relevant to usability analysis. For example, helpful self-assessments of performance, yielding insights into the impact of encountered difficulties. However, a small number of less desirable utterance types emerged: hypothesising, rationalising and forgetting. When used together, both methods contributed to our understanding of usability issues; the concurrent phase yielded more usability issues overall, the retrospective data improved our understanding of these by: (i) reinforcement: users highlighted the impact of an issue on their experience; (ii) elaboration: users would provide causal explanations of encountered difficulties; (iii) context: users provided information about the product's context of use.