Back in 1997 I wrote an article for the American Marketing Association concerning control issues in research design.  The AMA titled the article “Control is elusive in research design” and it is as pertinent today as it was all those years ago.

The article talks about the important role control plays in the college research lab – how students learn that the integrity of research findings rests in large measure on the level of controls built into the designs – and contrasts this to the different reality researchers find in the commercial world.   While understandably the role of controls in marketing research differs with the experimental intent in the lab, the ability to set controls within our designs is just as important to the success or failure of our research.  And yet defining and implementing parameters of control is elusive outside the research lab due to many factors, not the least of which are those related to the peculiarities of clients’ needs & resources, timeliness, logistics, and cost.  The difficulty of managing control can also come from the research process itself.

In quantitative research we strive to minimize error by, among other things, carefully designing our survey questionnaires to maximize clarity while minimizing potential response bias.  Yet, for all the effort put into questionnaire design and interviewer training, the reality of any given telephone study is that a certain amount of control must be relinquished when fieldwork begins – after all, it is impossible to train interviewers and coders for every situation or possibility, leaving them to make their own subjective judgments as necessary.

Qualitative researchers typically cry “oh no!” at the idea that controls play a part in qualitative designs, yet there are any number of control parameters built into these studies.  As one example, many moderators hope to gain a truer understanding of discussants’ reactions to visual stimuli by systematically controlling the order in which concept boards (whatever) are shown from focus group to focus group in a multi-group project.  Although I don’t agree with this practice (the reasons why may be explored in a subsequent post), it is interesting to note that the moderator’s defense of rotating stimuli typically draws on the quantitative-leaning concepts of primacy and recency.

And, of course, both quantitative and qualitative researchers grapple with lots of other control issues you won’t find in the research lab, such as professional respondents (“repeaters”) and those who pretend to be who they are not (“cheaters”).

So what is the role of control in our research designs?  Real world commercial research is not an ideal experimental environment but that shouldn’t diminish our consideration of control issues as we perfect our research designs.