Qualitative researchers are pretty good at distancing themselves from their quantitative colleagues, even to the point of bragging about their anything-goes right-brain sensibilities in contrast to the structured life of quantitative. So it boggles the mind when qual researchers so easily embrace certain quantitative concepts. One such concept is the randomization of stimuli in survey design in order to reduce primacy and recency effects.
I touched on this briefly in a February post when I discussed the relevance of cognitive-process theories in qualitative research (“Qualitative Research & Thinking About How People Think“). In that post, I argue that “the concept of primacy and recency effects are irrelevant in focus group research and, while randomizing the presentation order of stimuli is de rigueur in quantitative, not so in qualitative. To the contrary, I would suggest that not randomizing across group sessions adds a necessary component of control.”
So, why is a “component of control” important and how does that relate to whether the moderator randomizes or (more common in qualitative) rotates discussion stimuli such as concept boards? I have discussed the answer to this question in many private conversations with qualitative researchers who frankly have been quick to resist the idea of control in any form which of course stifles a true discussion. In one such conversation – after much talking around the issue – I ultimately resorted to a graphic, 4-slide depiction of my explanation. I present it here: