Paying Attention to Bias in Qualitative Research: A Message to Marketing Researchers (& Clients)

Researchers of all ilk care about bias and how it may creep into their research designs resulting in measurement error.  This is true among quantitative researchers as well as among qualitative researchers who routinely head-in-the-sand-2demonstrate their sensitivity to potential bias in their data by way of building interviewer training, careful recruitment screening, and appropriate modes into their research designs.  It is these types of measures that acknowledge qualitative researchers’ concerns about quality data; and yet, there are many other ways to mitigate bias in qualitative research that are often overlooked.

Marketing researchers (and marketing clients) in particular could benefit from thinking more deeply about bias and measurement error.  In the interest of “faster, cheaper, better” research solutions, marketing researchers often lose sight of quality design issues, not the least of which concern bias and measurement error in the data.  If marketing researchers care enough about mitigating bias to train interviewers/moderators, develop screening questions that effectively target the appropriate participant, and carefully select the suitable mode for the population segment, then it is sensible to adopt broader design standards that more fully embrace the collecting of quality data.

An example of a tool that serves to raise the design standard is the reflexive journal.  The reflexive journal has been the subject (in whole or in part) of many articles in Research Design Review, most notably “Interviewer Bias & Reflexivity in Qualitative Research” and “Reflections from the Field: Questions to Stimulate Reflexivity Among Qualitative Researchers”.   A reflexive journal is simply a diary of sorts that is utilized by the qualitative interviewer or moderator to think about (reflect on) how his/her assumptions or beliefs may be affecting the outcomes (i.e., the data).  It enables the researcher to re-assess (if necessary) his/her behavior, attitude, question wording, or other aspects of data collection for the purpose of mitigating distortions in the data.

The reflexive journal appears to be a particularly vague or foreign concept among qualitative marketing researchers (and marketing clients) given the absence of discussions concerning this tool in their research designs.  Why is this?  Is there a belief that interviewer/moderator training sufficiently guards against potential bias?  Is there a belief that all qualitative research is biased to some degree – because, after all, it isn’t survey research – so any attempt at mitigation is futile (which, of course, begs the question, ‘Why bother with qualitative research at all?’)?  Is there a head-in-the-sand (i.e., not-wanting-to-know) mentality that refuses to think of the interviewer/moderator as someone with assumptions, beliefs, values, and judgments but rather as a “super human” who is able to conduct a semi-structured in-depth interview (IDI) or focus group discussion devoid of these human qualities (i.e., lacking humanness)?

The humanness in all of us is worthy of reflection.  And in qualitative research design this reflection can be put to good use mitigating bias in our data.  As the interviewer considers how certain behavior may have elicited responses that were not true to the participant, or the moderator reflects on how his/her favoritism and attention towards a few focus group participants over others shifted the course of conversation and the outcomes of the discussion, these researchers are using their introspection to improve the research by moving data collection (and data outcomes) to a higher standard.  This is how interviewers learn to adjust the interview guide or consciously alter their behavior during an IDI to gain more accurate data, or the moderator comes to understand his/her own prejudices and finds corrective techniques to become a more inclusive moderator and ensure an even-handed approach to the discussion.

Two important and unique attributes to qualitative research methods are the “researcher as instrument” component, i.e., the researcher is the data collection tool, and the participant-researcher relationship.  These attributes speak to the humanness that both enriches and complicates the social-exchange environment of the IDI and focus group discussion.  And it is this humanness – embedded in qualitative research – that should obligate marketing researchers to consider its import in achieving quality outcomes.  If marketers care enough about the integrity of their data to adopt high standards in training, recruiting, and mode, why not care enough to mitigate bias in data collection by utilizing tools – such as a reflexive journal – to seriously examine the human factors that potentially increase inaccuracies and error in the final data?

Image captured from:


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.